[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:42:41 dist-test-slave-pv0w krb5kdc[9356](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:42:41 dist-test-slave-pv0w krb5kdc[9356](info): set up 2 sockets
Sep 28 14:42:41 dist-test-slave-pv0w krb5kdc[9356](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260928 14:42:42.604640 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:42:42 dist-test-slave-pv0w krb5kdc[9356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:42:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:42:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.33.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
I20260928 14:42:42.736912 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:34437
--webserver_interface=127.9.33.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:42041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:34437
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:42.821813 9369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:42.822084 9369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:42.822127 9369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:42.823503 9369 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:42:42.823614 9369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:42.823647 9369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:42.823683 9369 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:42:42.823707 9369 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:42:42.825469 9369 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:42041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:34437
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:34437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:42.825802 9369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:42.826071 9369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:42.831586 9377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:42.831588 9375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:42.831612 9369 server_base.cc:1034] running on GCE node
W20260928 14:42:42.831617 9374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:42.832118 9369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:42.832422 9369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:42.833611 9369 hybrid_clock.cc:648] HybridClock initialized: now 1790606562833585 us; error 59 us; skew 500 ppm
Sep 28 14:42:42 dist-test-slave-pv0w krb5kdc[9356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606562, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:42.836570 9369 init.cc:377] Logged in from keytab as kudu/127.9.33.126@KRBTEST.COM (short username kudu)
I20260928 14:42:42.837169 9369 webserver.cc:460] Webserver started at http://127.9.33.126:43437/ using document root <none> and password file <none>
I20260928 14:42:42.837463 9369 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:42.837545 9369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:42.837704 9369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:42.838892 9369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "23981139d85145f79ba2da1175e034d0"
format_stamp: "Formatted at 2026-09-28 14:42:42 on dist-test-slave-pv0w"
server_key: "e66e92c5656724325e047153beea4e6b"
server_key_iv: "8d4ea8742e0bb1a4b299265a89f05fec"
server_key_version: "encryptionkey@0"
I20260928 14:42:42.839290 9369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "23981139d85145f79ba2da1175e034d0"
format_stamp: "Formatted at 2026-09-28 14:42:42 on dist-test-slave-pv0w"
server_key: "e66e92c5656724325e047153beea4e6b"
server_key_iv: "8d4ea8742e0bb1a4b299265a89f05fec"
server_key_version: "encryptionkey@0"
I20260928 14:42:42.841074 9369 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260928 14:42:42.841934 9384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:42.842118 9369 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:42:42.842226 9369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "23981139d85145f79ba2da1175e034d0"
format_stamp: "Formatted at 2026-09-28 14:42:42 on dist-test-slave-pv0w"
server_key: "e66e92c5656724325e047153beea4e6b"
server_key_iv: "8d4ea8742e0bb1a4b299265a89f05fec"
server_key_version: "encryptionkey@0"
I20260928 14:42:42.842303 9369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:42.854543 9369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:42.857136 9369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:42.857259 9369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:42.861552 9369 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:34437
I20260928 14:42:42.861585 9436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:34437 every 8 connection(s)
I20260928 14:42:42.861980 9369 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:42:42.862461 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 9369
I20260928 14:42:42.862558 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:42:42.863046 9437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:42.863260 9349 external_mini_cluster.cc:1428] Setting key cc44b8ef4f4d0e18742e5b7994c06441
I20260928 14:42:42.866102 9437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23981139d85145f79ba2da1175e034d0: Bootstrap starting.
I20260928 14:42:42.866712 9437 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23981139d85145f79ba2da1175e034d0: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:42.867048 9437 log.cc:824] T 00000000000000000000000000000000 P 23981139d85145f79ba2da1175e034d0: Log is configured to *not* fsync() on all Append() calls
I20260928 14:42:42.867934 9437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23981139d85145f79ba2da1175e034d0: No bootstrap required, opened a new log
I20260928 14:42:42.870417 9437 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23981139d85145f79ba2da1175e034d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23981139d85145f79ba2da1175e034d0" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 34437 } }
I20260928 14:42:42.870625 9437 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23981139d85145f79ba2da1175e034d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:42.870666 9437 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23981139d85145f79ba2da1175e034d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23981139d85145f79ba2da1175e034d0, State: Initialized, Role: FOLLOWER
I20260928 14:42:42.870818 9437 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23981139d85145f79ba2da1175e034d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23981139d85145f79ba2da1175e034d0" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 34437 } }
I20260928 14:42:42.870905 9437 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23981139d85145f79ba2da1175e034d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:42:42.870937 9437 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23981139d85145f79ba2da1175e034d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:42:42.870985 9437 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23981139d85145f79ba2da1175e034d0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:42.871780 9437 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23981139d85145f79ba2da1175e034d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23981139d85145f79ba2da1175e034d0" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 34437 } }
I20260928 14:42:42.871899 9437 leader_election.cc:302] T 00000000000000000000000000000000 P 23981139d85145f79ba2da1175e034d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23981139d85145f79ba2da1175e034d0; no voters:
I20260928 14:42:42.872093 9437 leader_election.cc:288] T 00000000000000000000000000000000 P 23981139d85145f79ba2da1175e034d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:42:42.872146 9442 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23981139d85145f79ba2da1175e034d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:42:42.872324 9442 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23981139d85145f79ba2da1175e034d0 [term 1 LEADER]: Becoming Leader. State: Replica: 23981139d85145f79ba2da1175e034d0, State: Running, Role: LEADER
I20260928 14:42:42.872416 9442 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23981139d85145f79ba2da1175e034d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23981139d85145f79ba2da1175e034d0" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 34437 } }
I20260928 14:42:42.872480 9437 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23981139d85145f79ba2da1175e034d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:42:42.872828 9444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23981139d85145f79ba2da1175e034d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23981139d85145f79ba2da1175e034d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23981139d85145f79ba2da1175e034d0" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 34437 } } }
I20260928 14:42:42.872923 9444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23981139d85145f79ba2da1175e034d0 [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:42.873070 9444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23981139d85145f79ba2da1175e034d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23981139d85145f79ba2da1175e034d0. Latest consensus state: current_term: 1 leader_uuid: "23981139d85145f79ba2da1175e034d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23981139d85145f79ba2da1175e034d0" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 34437 } } }
I20260928 14:42:42.873127 9444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23981139d85145f79ba2da1175e034d0 [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:42.873268 9448 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:42:42.874034 9448 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:42:42.875866 9448 catalog_manager.cc:1347] Generated new cluster ID: df02b4b407c24316be4685aa17eea70d
I20260928 14:42:42.875922 9448 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 14:42:42 dist-test-slave-pv0w krb5kdc[9356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:42:42.882230 9439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:42.864224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47928 (local address 127.9.33.126:34437)
0928 14:42:42.864402 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:42.864413 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:42.864664 (+ 251us) server_negotiation.cc:407] Connection header received
0928 14:42:42.867011 (+ 2347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:42.867018 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:42.867067 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:42.867233 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:42.869500 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:42.870443 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:42.872095 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:42.872770 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:42.873970 (+ 1200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:42.877669 (+ 3699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:42.877674 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:42.877674 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:42.877688 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:42.879812 (+ 2124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:42.880399 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:42.880403 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:42.880403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:42.880531 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:42.880842 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:42.880844 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:42.880844 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:42.881268 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:42.881455 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:42.881753 (+ 298us) server_negotiation.cc:299] Negotiation successful
0928 14:42:42.881999 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
I20260928 14:42:42.896315 9448 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:42:42.897084 9448 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:42:42.905393 9448 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23981139d85145f79ba2da1175e034d0: Generated new TSK 0
I20260928 14:42:42.905628 9448 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.33.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab.
Entry for principal kudu/127.9.33.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab.
I20260928 14:42:42.940444 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.65:0
--local_ip_for_outbound_sockets=127.9.33.65
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:34437
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:42041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:43.024792 9463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:43.025053 9463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:43.025098 9463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:43.026350 9463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:43.026427 9463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:43.026477 9463 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.65
I20260928 14:42:43.028272 9463 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:42041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:34437
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9463
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.65
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:43.028558 9463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:43.028807 9463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:43.034458 9471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:43.034439 9463 server_base.cc:1034] running on GCE node
W20260928 14:42:43.034409 9468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:43.034403 9469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:43.034930 9463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:43.035218 9463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:43.036403 9463 hybrid_clock.cc:648] HybridClock initialized: now 1790606563036375 us; error 43 us; skew 500 ppm
Sep 28 14:42:43 dist-test-slave-pv0w krb5kdc[9356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606563, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:43.039156 9463 init.cc:377] Logged in from keytab as kudu/127.9.33.65@KRBTEST.COM (short username kudu)
I20260928 14:42:43.039644 9463 webserver.cc:460] Webserver started at http://127.9.33.65:41323/ using document root <none> and password file <none>
I20260928 14:42:43.039918 9463 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:43.039994 9463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:43.040160 9463 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:43.041370 9463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-0/data/instance:
uuid: "d3597c33eed34d05b414f9b0b20fd929"
format_stamp: "Formatted at 2026-09-28 14:42:43 on dist-test-slave-pv0w"
server_key: "0b8253f1bb5e4ff28f71228bcdd0fb6e"
server_key_iv: "73818b625b51b25fc8e98c6f81d35d45"
server_key_version: "encryptionkey@0"
I20260928 14:42:43.041805 9463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance:
uuid: "d3597c33eed34d05b414f9b0b20fd929"
format_stamp: "Formatted at 2026-09-28 14:42:43 on dist-test-slave-pv0w"
server_key: "0b8253f1bb5e4ff28f71228bcdd0fb6e"
server_key_iv: "73818b625b51b25fc8e98c6f81d35d45"
server_key_version: "encryptionkey@0"
I20260928 14:42:43.043426 9463 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:42:43.044373 9478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:43.044624 9463 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:42:43.044739 9463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-0/wal
uuid: "d3597c33eed34d05b414f9b0b20fd929"
format_stamp: "Formatted at 2026-09-28 14:42:43 on dist-test-slave-pv0w"
server_key: "0b8253f1bb5e4ff28f71228bcdd0fb6e"
server_key_iv: "73818b625b51b25fc8e98c6f81d35d45"
server_key_version: "encryptionkey@0"
I20260928 14:42:43.044862 9463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:43.062247 9463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:43.064613 9463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:43.064792 9463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:43.065058 9463 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:42:43.065425 9463 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:42:43.065480 9463 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:43.065515 9463 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:42:43.065546 9463 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:43.071506 9463 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.65:39821
I20260928 14:42:43.071551 9591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.65:39821 every 8 connection(s)
I20260928 14:42:43.072036 9463 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
I20260928 14:42:43.075873 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 9463
I20260928 14:42:43.075977 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance
I20260928 14:42:43.076148 9349 external_mini_cluster.cc:1428] Setting key 21a879db917465d8a55b08a1e7fad144
Sep 28 14:42:43 dist-test-slave-pv0w krb5kdc[9356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606563, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:42:43.082813 9439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:43.072779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:48321 (local address 127.9.33.126:34437)
0928 14:42:43.072827 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:43.072829 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:43.073129 (+ 300us) server_negotiation.cc:407] Connection header received
0928 14:42:43.073634 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:43.073635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:43.073645 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:43.073697 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:43.074796 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:43.075299 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:43.076736 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:43.077049 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:43.077139 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:43.079348 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:43.079350 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:43.079350 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:43.079359 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:43.081149 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:43.081570 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:43.081571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:43.081572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:43.081640 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:43.082035 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:43.082036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:43.082036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:43.082208 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:43.082292 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:43.082574 (+ 282us) server_negotiation.cc:299] Negotiation successful
0928 14:42:43.082690 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 14:42:43.082839 9594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:43.072757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:34437 (local address 127.9.33.65:48321)
0928 14:42:43.072937 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:43.072974 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:43.073540 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:43.073756 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:43.073759 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:43.074433 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:43.074654 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:43.074660 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:43.075398 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:43.075399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:43.076638 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:43.076640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:43.077149 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:43.077150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:43.077293 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:43.077874 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:43.077883 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:43.079257 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:43.081238 (+ 1981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:43.081245 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:43.081247 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:43.081488 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:43.081744 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:43.081745 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:43.081745 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:43.081952 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:43.082290 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:43.082296 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:43.082430 (+ 134us) client_negotiation.cc:769] Sending connection context
0928 14:42:43.082522 (+ 92us) client_negotiation.cc:240] Negotiation successful
0928 14:42:43.082667 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":94,"threads_started":1}
I20260928 14:42:43.083432 9592 heartbeater.cc:347] Connected to a master server at 127.9.33.126:34437
I20260928 14:42:43.083524 9592 heartbeater.cc:464] Registering TS with master...
I20260928 14:42:43.083791 9592 heartbeater.cc:511] Master 127.9.33.126:34437 requested a full tablet report, sending...
I20260928 14:42:43.084376 9401 ts_manager.cc:194] Registered new tserver with Master: d3597c33eed34d05b414f9b0b20fd929 (127.9.33.65:39821)
I20260928 14:42:43.084987 9401 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.65@KRBTEST.COM'} at 127.9.33.65:48321
WARNING: no policy specified for kudu/127.9.33.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab.
Entry for principal kudu/127.9.33.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab.
I20260928 14:42:43.106964 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.66:0
--local_ip_for_outbound_sockets=127.9.33.66
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:34437
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:42041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:43.209880 9597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:43.210143 9597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:43.210186 9597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:43.211366 9597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:43.211434 9597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:43.211481 9597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.66
I20260928 14:42:43.213071 9597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:42041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:34437
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.66
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:43.213351 9597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:43.213588 9597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:43.218778 9602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:43.218894 9597 server_base.cc:1034] running on GCE node
W20260928 14:42:43.218791 9605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:43.218791 9603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:43.219220 9597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:43.219484 9597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:43.220660 9597 hybrid_clock.cc:648] HybridClock initialized: now 1790606563220651 us; error 34 us; skew 500 ppm
Sep 28 14:42:43 dist-test-slave-pv0w krb5kdc[9356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606563, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:43.223237 9597 init.cc:377] Logged in from keytab as kudu/127.9.33.66@KRBTEST.COM (short username kudu)
I20260928 14:42:43.223656 9597 webserver.cc:460] Webserver started at http://127.9.33.66:39027/ using document root <none> and password file <none>
I20260928 14:42:43.223912 9597 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:43.223960 9597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:43.224053 9597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:43.225314 9597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-1/data/instance:
uuid: "ecddc87f04a24ea68bfa226d8a3a4ff4"
format_stamp: "Formatted at 2026-09-28 14:42:43 on dist-test-slave-pv0w"
server_key: "a5123ad3d5348fd3d47ce41dc6cf9abd"
server_key_iv: "5f13478dfe15eebaef92c492391ca538"
server_key_version: "encryptionkey@0"
I20260928 14:42:43.225646 9597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance:
uuid: "ecddc87f04a24ea68bfa226d8a3a4ff4"
format_stamp: "Formatted at 2026-09-28 14:42:43 on dist-test-slave-pv0w"
server_key: "a5123ad3d5348fd3d47ce41dc6cf9abd"
server_key_iv: "5f13478dfe15eebaef92c492391ca538"
server_key_version: "encryptionkey@0"
I20260928 14:42:43.227171 9597 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:42:43.228111 9612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:43.228356 9597 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:42:43.228507 9597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-1/wal
uuid: "ecddc87f04a24ea68bfa226d8a3a4ff4"
format_stamp: "Formatted at 2026-09-28 14:42:43 on dist-test-slave-pv0w"
server_key: "a5123ad3d5348fd3d47ce41dc6cf9abd"
server_key_iv: "5f13478dfe15eebaef92c492391ca538"
server_key_version: "encryptionkey@0"
I20260928 14:42:43.228585 9597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:43.249322 9597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:43.251963 9597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:43.252130 9597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:43.252435 9597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:42:43.252802 9597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:42:43.252839 9597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:43.252862 9597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:42:43.252875 9597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:43.259003 9597 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.66:33453
I20260928 14:42:43.259104 9725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.66:33453 every 8 connection(s)
I20260928 14:42:43.259439 9597 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
I20260928 14:42:43.262667 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 9597
I20260928 14:42:43.262771 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance
I20260928 14:42:43.262930 9349 external_mini_cluster.cc:1428] Setting key 8f3810f9ff1ea5f9fe56ce37ece5b097
Sep 28 14:42:43 dist-test-slave-pv0w krb5kdc[9356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606563, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:42:43.270377 9439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:43.260304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:41059 (local address 127.9.33.126:34437)
0928 14:42:43.260380 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:43.260382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:43.260630 (+ 248us) server_negotiation.cc:407] Connection header received
0928 14:42:43.261167 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:43.261168 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:43.261179 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:43.261227 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:43.262256 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:43.262981 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:43.264275 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:43.264614 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:43.264690 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:43.267203 (+ 2513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:43.267206 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:43.267206 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:43.267214 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:43.268878 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:43.269354 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:43.269356 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:43.269356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:43.269425 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:43.269688 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:43.269689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:43.269689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:43.269820 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:43.269899 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:43.270185 (+ 286us) server_negotiation.cc:299] Negotiation successful
0928 14:42:43.270278 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260928 14:42:43.270589 9728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:43.260292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:34437 (local address 127.9.33.66:41059)
0928 14:42:43.260492 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:43.260529 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:43.261075 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:43.261312 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:43.261317 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:43.261911 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:43.262141 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:43.262147 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:43.263104 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:43.263105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:43.264182 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:43.264185 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:43.264743 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:43.264744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:43.264894 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:43.265488 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:43.265497 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:43.267093 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:43.269000 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:43.269007 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:43.269010 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:43.269248 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:43.269497 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:43.269497 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:43.269498 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:43.269618 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:43.269890 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:43.269897 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:43.270049 (+ 152us) client_negotiation.cc:769] Sending connection context
0928 14:42:43.270164 (+ 115us) client_negotiation.cc:240] Negotiation successful
0928 14:42:43.270345 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":97,"threads_started":1}
WARNING: no policy specified for kudu/127.9.33.67@KRBTEST.COM; defaulting to no policy
I20260928 14:42:43.271327 9726 heartbeater.cc:347] Connected to a master server at 127.9.33.126:34437
I20260928 14:42:43.271456 9726 heartbeater.cc:464] Registering TS with master...
I20260928 14:42:43.271642 9726 heartbeater.cc:511] Master 127.9.33.126:34437 requested a full tablet report, sending...
I20260928 14:42:43.272116 9401 ts_manager.cc:194] Registered new tserver with Master: ecddc87f04a24ea68bfa226d8a3a4ff4 (127.9.33.66:33453)
I20260928 14:42:43.272608 9401 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.66@KRBTEST.COM'} at 127.9.33.66:41059
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab.
Entry for principal kudu/127.9.33.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab.
I20260928 14:42:43.293016 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.67:0
--local_ip_for_outbound_sockets=127.9.33.67
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:34437
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:42041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:43.397799 9731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:43.398181 9731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:43.398255 9731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:43.400919 9731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:43.401048 9731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:43.401149 9731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.67
I20260928 14:42:43.404107 9731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:42041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:34437
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9731
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.67
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:43.404599 9731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:43.404893 9731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:43.410408 9736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:43.410435 9731 server_base.cc:1034] running on GCE node
W20260928 14:42:43.410436 9737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:43.410415 9739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:43.410755 9731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:43.411003 9731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:43.412161 9731 hybrid_clock.cc:648] HybridClock initialized: now 1790606563412147 us; error 29 us; skew 500 ppm
Sep 28 14:42:43 dist-test-slave-pv0w krb5kdc[9356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606563, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:43.414655 9731 init.cc:377] Logged in from keytab as kudu/127.9.33.67@KRBTEST.COM (short username kudu)
I20260928 14:42:43.415194 9731 webserver.cc:460] Webserver started at http://127.9.33.67:40381/ using document root <none> and password file <none>
I20260928 14:42:43.415385 9731 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:43.415426 9731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:43.415529 9731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:43.416832 9731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-2/data/instance:
uuid: "b4feb84dcb5e4b5b9714f9702adc1480"
format_stamp: "Formatted at 2026-09-28 14:42:43 on dist-test-slave-pv0w"
server_key: "1a8584c7764e744f3f8ad5b97d3fcd86"
server_key_iv: "51ade8b76fc4c5009fcd8b217c5f5941"
server_key_version: "encryptionkey@0"
I20260928 14:42:43.417234 9731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance:
uuid: "b4feb84dcb5e4b5b9714f9702adc1480"
format_stamp: "Formatted at 2026-09-28 14:42:43 on dist-test-slave-pv0w"
server_key: "1a8584c7764e744f3f8ad5b97d3fcd86"
server_key_iv: "51ade8b76fc4c5009fcd8b217c5f5941"
server_key_version: "encryptionkey@0"
I20260928 14:42:43.418676 9731 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:42:43.419499 9746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:43.419922 9731 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:42:43.420003 9731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-2/wal
uuid: "b4feb84dcb5e4b5b9714f9702adc1480"
format_stamp: "Formatted at 2026-09-28 14:42:43 on dist-test-slave-pv0w"
server_key: "1a8584c7764e744f3f8ad5b97d3fcd86"
server_key_iv: "51ade8b76fc4c5009fcd8b217c5f5941"
server_key_version: "encryptionkey@0"
I20260928 14:42:43.420099 9731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:43.437163 9731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:43.439127 9731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:43.439294 9731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:43.439543 9731 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:42:43.439924 9731 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:42:43.439961 9731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:43.439985 9731 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:42:43.440032 9731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:43.445941 9731 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.67:44831
I20260928 14:42:43.446035 9859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.67:44831 every 8 connection(s)
I20260928 14:42:43.446369 9731 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
I20260928 14:42:43.448273 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 9731
I20260928 14:42:43.448347 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance
I20260928 14:42:43.448505 9349 external_mini_cluster.cc:1428] Setting key 30afaeed5c645e6515a0ff935715e7ac
Sep 28 14:42:43 dist-test-slave-pv0w krb5kdc[9356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606563, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:42:43.455978 9439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:43.447100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:52515 (local address 127.9.33.126:34437)
0928 14:42:43.447180 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:43.447182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:43.447389 (+ 207us) server_negotiation.cc:407] Connection header received
0928 14:42:43.448027 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:43.448028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:43.448039 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:43.448092 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:43.449374 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:43.450033 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:43.450998 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:43.451186 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:43.451240 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:43.453508 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:43.453509 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:43.453509 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:43.453516 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:43.454706 (+ 1190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:43.455070 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:43.455071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:43.455072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:43.455121 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:43.455324 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:43.455325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:43.455325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:43.455424 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:43.455474 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:43.455849 (+ 375us) server_negotiation.cc:299] Negotiation successful
0928 14:42:43.455914 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260928 14:42:43.456264 9862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:43.447078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:34437 (local address 127.9.33.67:52515)
0928 14:42:43.447260 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:43.447295 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:43.447938 (+ 643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:43.448162 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:43.448166 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:43.448965 (+ 799us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:43.449266 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:43.449274 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:43.450115 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:43.450116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:43.450919 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:43.450921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:43.451279 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:43.451280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:43.451392 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:43.451997 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:43.452005 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:43.453427 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:43.454782 (+ 1355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:43.454788 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:43.454789 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:43.455005 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:43.455178 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:43.455179 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:43.455179 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:43.455271 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:43.455510 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:43.455517 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:43.455639 (+ 122us) client_negotiation.cc:769] Sending connection context
0928 14:42:43.455832 (+ 193us) client_negotiation.cc:240] Negotiation successful
0928 14:42:43.455970 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":103,"threads_started":1}
I20260928 14:42:43.456839 9860 heartbeater.cc:347] Connected to a master server at 127.9.33.126:34437
I20260928 14:42:43.456971 9860 heartbeater.cc:464] Registering TS with master...
I20260928 14:42:43.457175 9860 heartbeater.cc:511] Master 127.9.33.126:34437 requested a full tablet report, sending...
I20260928 14:42:43.457609 9401 ts_manager.cc:194] Registered new tserver with Master: b4feb84dcb5e4b5b9714f9702adc1480 (127.9.33.67:44831)
I20260928 14:42:43.458022 9401 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.67@KRBTEST.COM'} at 127.9.33.67:52515
I20260928 14:42:43.460538 9349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 28 14:42:43 dist-test-slave-pv0w krb5kdc[9356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606563, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 28 14:42:43 dist-test-slave-pv0w krb5kdc[9356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606563, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.33.65@KRBTEST.COM
I20260928 14:42:43.484027 9867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:43.476518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55422 (local address 127.9.33.65:39821)
0928 14:42:43.476699 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:43.476704 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:43.476714 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:42:43.476743 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:43.476745 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:43.476759 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:43.476831 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:43.477385 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:43.478142 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:43.478890 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:43.479195 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:43.479249 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:43.481400 (+ 2151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:43.481408 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:43.481408 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:43.481415 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:43.482937 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:43.483244 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:43.483247 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:43.483247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:43.483300 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:43.483459 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:43.483460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:43.483460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:43.483585 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:43.483686 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:43.483843 (+ 157us) server_negotiation.cc:299] Negotiation successful
0928 14:42:43.483915 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":103,"threads_started":1}
W20260928 14:42:43.484449 9506 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55422
Sep 28 14:42:43 dist-test-slave-pv0w krb5kdc[9356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 14:42:43 dist-test-slave-pv0w krb5kdc[9356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.65@KRBTEST.COM
I20260928 14:42:43.507201 9867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:43.499655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55438 (local address 127.9.33.65:39821)
0928 14:42:43.499781 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:43.499783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:43.499876 (+ 93us) server_negotiation.cc:407] Connection header received
0928 14:42:43.499969 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:43.499970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:43.499990 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:43.500051 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:43.500650 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:43.501189 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:43.501965 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:43.502182 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:43.502269 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:43.504673 (+ 2404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:43.504675 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:43.504675 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:43.504684 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:43.506068 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:43.506421 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:43.506422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:43.506423 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:43.506481 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:43.506683 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:43.506684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:43.506684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:43.506790 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:43.506921 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:43.507012 (+ 91us) server_negotiation.cc:299] Negotiation successful
0928 14:42:43.507093 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260928 14:42:43.508039 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 9463
I20260928 14:42:43.512954 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 9597
I20260928 14:42:43.518440 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 9731
I20260928 14:42:43.523180 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 9369
2026-09-28T14:42:43Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1985 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:42:43 dist-test-slave-pv0w krb5kdc[9874](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:42:43 dist-test-slave-pv0w krb5kdc[9874](info): set up 2 sockets
Sep 28 14:42:43 dist-test-slave-pv0w krb5kdc[9874](info): commencing operation
krb5kdc: starting...
W20260928 14:42:44.560113 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:42:44 dist-test-slave-pv0w krb5kdc[9874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606564, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:42:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:42:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.33.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
I20260928 14:42:44.690680 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:44591
--webserver_interface=127.9.33.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:35605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:44591
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:44.775110 9887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:44.775336 9887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:44.775360 9887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:44.776561 9887 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:42:44.776604 9887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:44.776618 9887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:44.776631 9887 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:42:44.776643 9887 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:42:44.778121 9887 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:35605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:44591
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:44591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:44.778334 9887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:44.778514 9887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:44.783636 9892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:44.783612 9895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:44.783896 9887 server_base.cc:1034] running on GCE node
W20260928 14:42:44.783877 9893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:44.784231 9887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:44.784583 9887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:44.785833 9887 hybrid_clock.cc:648] HybridClock initialized: now 1790606564785810 us; error 47 us; skew 500 ppm
Sep 28 14:42:44 dist-test-slave-pv0w krb5kdc[9874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606564, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:44.788662 9887 init.cc:377] Logged in from keytab as kudu/127.9.33.126@KRBTEST.COM (short username kudu)
I20260928 14:42:44.789181 9887 webserver.cc:460] Webserver started at http://127.9.33.126:33891/ using document root <none> and password file <none>
I20260928 14:42:44.789426 9887 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:44.789499 9887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:44.789654 9887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:44.790822 9887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "e64d01f5e1d74821bad40e9a016ca345"
format_stamp: "Formatted at 2026-09-28 14:42:44 on dist-test-slave-pv0w"
server_key: "d07b207383ef681c22904af79fd174bb"
server_key_iv: "0df154c2afcd9e602719b4f9ad7a5a47"
server_key_version: "encryptionkey@0"
I20260928 14:42:44.791200 9887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "e64d01f5e1d74821bad40e9a016ca345"
format_stamp: "Formatted at 2026-09-28 14:42:44 on dist-test-slave-pv0w"
server_key: "d07b207383ef681c22904af79fd174bb"
server_key_iv: "0df154c2afcd9e602719b4f9ad7a5a47"
server_key_version: "encryptionkey@0"
I20260928 14:42:44.792824 9887 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:42:44.793747 9902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:44.794011 9887 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:42:44.794145 9887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "e64d01f5e1d74821bad40e9a016ca345"
format_stamp: "Formatted at 2026-09-28 14:42:44 on dist-test-slave-pv0w"
server_key: "d07b207383ef681c22904af79fd174bb"
server_key_iv: "0df154c2afcd9e602719b4f9ad7a5a47"
server_key_version: "encryptionkey@0"
I20260928 14:42:44.794239 9887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:44.820122 9887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:44.822963 9887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:44.823166 9887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:44.827893 9887 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:44591
I20260928 14:42:44.827935 9954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:44591 every 8 connection(s)
I20260928 14:42:44.828341 9887 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:42:44.829267 9955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:44.831845 9955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e64d01f5e1d74821bad40e9a016ca345: Bootstrap starting.
I20260928 14:42:44.832479 9955 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e64d01f5e1d74821bad40e9a016ca345: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:44.832803 9955 log.cc:824] T 00000000000000000000000000000000 P e64d01f5e1d74821bad40e9a016ca345: Log is configured to *not* fsync() on all Append() calls
I20260928 14:42:44.833583 9955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e64d01f5e1d74821bad40e9a016ca345: No bootstrap required, opened a new log
I20260928 14:42:44.835330 9955 raft_consensus.cc:348] T 00000000000000000000000000000000 P e64d01f5e1d74821bad40e9a016ca345 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e64d01f5e1d74821bad40e9a016ca345" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 44591 } }
I20260928 14:42:44.835484 9955 raft_consensus.cc:374] T 00000000000000000000000000000000 P e64d01f5e1d74821bad40e9a016ca345 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:44.835506 9955 raft_consensus.cc:729] T 00000000000000000000000000000000 P e64d01f5e1d74821bad40e9a016ca345 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e64d01f5e1d74821bad40e9a016ca345, State: Initialized, Role: FOLLOWER
I20260928 14:42:44.835675 9955 consensus_queue.cc:260] T 00000000000000000000000000000000 P e64d01f5e1d74821bad40e9a016ca345 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e64d01f5e1d74821bad40e9a016ca345" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 44591 } }
I20260928 14:42:44.835809 9955 raft_consensus.cc:388] T 00000000000000000000000000000000 P e64d01f5e1d74821bad40e9a016ca345 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:42:44.835891 9955 raft_consensus.cc:482] T 00000000000000000000000000000000 P e64d01f5e1d74821bad40e9a016ca345 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:42:44.835934 9955 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e64d01f5e1d74821bad40e9a016ca345 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:44.836079 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 9887
I20260928 14:42:44.836176 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:42:44.836334 9349 external_mini_cluster.cc:1428] Setting key fa510a59a9c5423608ba60ddb5fb5e91
I20260928 14:42:44.836694 9955 raft_consensus.cc:504] T 00000000000000000000000000000000 P e64d01f5e1d74821bad40e9a016ca345 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e64d01f5e1d74821bad40e9a016ca345" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 44591 } }
I20260928 14:42:44.836834 9955 leader_election.cc:302] T 00000000000000000000000000000000 P e64d01f5e1d74821bad40e9a016ca345 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e64d01f5e1d74821bad40e9a016ca345; no voters:
I20260928 14:42:44.837029 9955 leader_election.cc:288] T 00000000000000000000000000000000 P e64d01f5e1d74821bad40e9a016ca345 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:42:44.837306 9959 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e64d01f5e1d74821bad40e9a016ca345 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:42:44.837311 9955 sys_catalog.cc:564] T 00000000000000000000000000000000 P e64d01f5e1d74821bad40e9a016ca345 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:42:44.837569 9959 raft_consensus.cc:686] T 00000000000000000000000000000000 P e64d01f5e1d74821bad40e9a016ca345 [term 1 LEADER]: Becoming Leader. State: Replica: e64d01f5e1d74821bad40e9a016ca345, State: Running, Role: LEADER
I20260928 14:42:44.837809 9959 consensus_queue.cc:237] T 00000000000000000000000000000000 P e64d01f5e1d74821bad40e9a016ca345 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e64d01f5e1d74821bad40e9a016ca345" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 44591 } }
I20260928 14:42:44.838382 9962 sys_catalog.cc:455] T 00000000000000000000000000000000 P e64d01f5e1d74821bad40e9a016ca345 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e64d01f5e1d74821bad40e9a016ca345. Latest consensus state: current_term: 1 leader_uuid: "e64d01f5e1d74821bad40e9a016ca345" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e64d01f5e1d74821bad40e9a016ca345" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 44591 } } }
I20260928 14:42:44.838543 9962 sys_catalog.cc:458] T 00000000000000000000000000000000 P e64d01f5e1d74821bad40e9a016ca345 [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:44.838555 9961 sys_catalog.cc:455] T 00000000000000000000000000000000 P e64d01f5e1d74821bad40e9a016ca345 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e64d01f5e1d74821bad40e9a016ca345" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e64d01f5e1d74821bad40e9a016ca345" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 44591 } } }
I20260928 14:42:44.838773 9961 sys_catalog.cc:458] T 00000000000000000000000000000000 P e64d01f5e1d74821bad40e9a016ca345 [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:44.839154 9971 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:42:44.839977 9971 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:42:44.844117 9971 catalog_manager.cc:1347] Generated new cluster ID: a00f7bc19aed4a099c64587fb6e91dbe
I20260928 14:42:44.844182 9971 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 14:42:44 dist-test-slave-pv0w krb5kdc[9874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606564, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:42:44.849448 9960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:44.836936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58570 (local address 127.9.33.126:44591)
0928 14:42:44.837184 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:44.837195 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:44.837256 (+ 61us) server_negotiation.cc:407] Connection header received
0928 14:42:44.837431 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:44.837439 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:44.837516 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:44.837678 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:44.839625 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:44.841177 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:44.841987 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:44.842469 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:44.842596 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:44.845254 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:44.845260 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:44.845261 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:44.845276 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:44.847456 (+ 2180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:44.847905 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:44.847909 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:44.847909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:44.848042 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:44.848257 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:44.848258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:44.848258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:44.848631 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:44.848816 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:44.848897 (+ 81us) server_negotiation.cc:299] Negotiation successful
0928 14:42:44.849170 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":98,"threads_started":1}
I20260928 14:42:44.851147 9971 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:42:44.852022 9971 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:42:44.856937 9971 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e64d01f5e1d74821bad40e9a016ca345: Generated new TSK 0
I20260928 14:42:44.857136 9971 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.33.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab.
Entry for principal kudu/127.9.33.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab.
I20260928 14:42:44.887838 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.65:0
--local_ip_for_outbound_sockets=127.9.33.65
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:44591
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:35605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:44.970985 9981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:44.971204 9981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:44.971230 9981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:44.972543 9981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:44.972587 9981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:44.972640 9981 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.65
I20260928 14:42:44.974200 9981 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:35605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:44591
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.65
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:44.974466 9981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:44.974725 9981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:44.980187 9989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:44.980216 9987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:44.980247 9981 server_base.cc:1034] running on GCE node
W20260928 14:42:44.980196 9986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:44.980628 9981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:44.980899 9981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:44.982090 9981 hybrid_clock.cc:648] HybridClock initialized: now 1790606564982078 us; error 29 us; skew 500 ppm
Sep 28 14:42:44 dist-test-slave-pv0w krb5kdc[9874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606564, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:44.985149 9981 init.cc:377] Logged in from keytab as kudu/127.9.33.65@KRBTEST.COM (short username kudu)
I20260928 14:42:44.985736 9981 webserver.cc:460] Webserver started at http://127.9.33.65:34849/ using document root <none> and password file <none>
I20260928 14:42:44.985960 9981 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:44.986006 9981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:44.986107 9981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:44.987310 9981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-0/data/instance:
uuid: "04607244c7bd4521b011e1021cfc3912"
format_stamp: "Formatted at 2026-09-28 14:42:44 on dist-test-slave-pv0w"
server_key: "f8038093d7e95826dea95c66a06d6120"
server_key_iv: "088a57d32f5434b175b98b4ce4b38750"
server_key_version: "encryptionkey@0"
I20260928 14:42:44.987658 9981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance:
uuid: "04607244c7bd4521b011e1021cfc3912"
format_stamp: "Formatted at 2026-09-28 14:42:44 on dist-test-slave-pv0w"
server_key: "f8038093d7e95826dea95c66a06d6120"
server_key_iv: "088a57d32f5434b175b98b4ce4b38750"
server_key_version: "encryptionkey@0"
I20260928 14:42:44.989374 9981 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:42:44.990203 9996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:44.990433 9981 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:42:44.990523 9981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-0/wal
uuid: "04607244c7bd4521b011e1021cfc3912"
format_stamp: "Formatted at 2026-09-28 14:42:44 on dist-test-slave-pv0w"
server_key: "f8038093d7e95826dea95c66a06d6120"
server_key_iv: "088a57d32f5434b175b98b4ce4b38750"
server_key_version: "encryptionkey@0"
I20260928 14:42:44.990585 9981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:45.014540 9981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:45.016816 9981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:45.016955 9981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:45.017158 9981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:42:45.017527 9981 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:42:45.017565 9981 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:45.017587 9981 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:42:45.017638 9981 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:45.023408 9981 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.65:34369
I20260928 14:42:45.023481 10109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.65:34369 every 8 connection(s)
I20260928 14:42:45.023869 9981 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
Sep 28 14:42:45 dist-test-slave-pv0w krb5kdc[9874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606564, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:42:45.033330 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 9981
I20260928 14:42:45.033439 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance
I20260928 14:42:45.033627 9349 external_mini_cluster.cc:1428] Setting key d229aab9fdc3720cf483764c8a474b0a
I20260928 14:42:45.034371 9960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:45.024742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:43041 (local address 127.9.33.126:44591)
0928 14:42:45.024816 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:45.024818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:45.025055 (+ 237us) server_negotiation.cc:407] Connection header received
0928 14:42:45.026053 (+ 998us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:45.026054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:45.026069 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:45.026126 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:45.026795 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.027317 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.028505 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.028750 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.028813 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:45.031135 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:45.031136 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:45.031137 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:45.031145 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:45.032714 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:45.033113 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:45.033152 (+ 39us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:45.033152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:45.033223 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:45.033570 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:45.033571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:45.033572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:45.033742 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:45.033830 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:45.034186 (+ 356us) server_negotiation.cc:299] Negotiation successful
0928 14:42:45.034280 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260928 14:42:45.034596 10112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:45.024702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:44591 (local address 127.9.33.65:43041)
0928 14:42:45.024911 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:45.024948 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:45.025940 (+ 992us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:45.026210 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:45.026213 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:45.026493 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:45.026695 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:45.026699 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.027441 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.027442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:45.028406 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:45.028409 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.028835 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.028836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:45.028981 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:45.029543 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:45.029552 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:45.031025 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:45.032810 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:45.032817 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:45.032819 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:45.033046 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:45.033331 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:45.033332 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:45.033332 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:45.033476 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:45.033820 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:45.033827 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:45.034043 (+ 216us) client_negotiation.cc:769] Sending connection context
0928 14:42:45.034162 (+ 119us) client_negotiation.cc:240] Negotiation successful
0928 14:42:45.034336 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":83,"threads_started":1}
I20260928 14:42:45.035211 10110 heartbeater.cc:347] Connected to a master server at 127.9.33.126:44591
I20260928 14:42:45.035313 10110 heartbeater.cc:464] Registering TS with master...
I20260928 14:42:45.035519 10110 heartbeater.cc:511] Master 127.9.33.126:44591 requested a full tablet report, sending...
I20260928 14:42:45.036132 9919 ts_manager.cc:194] Registered new tserver with Master: 04607244c7bd4521b011e1021cfc3912 (127.9.33.65:34369)
I20260928 14:42:45.037106 9919 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.65@KRBTEST.COM'} at 127.9.33.65:43041
WARNING: no policy specified for kudu/127.9.33.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab.
Entry for principal kudu/127.9.33.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab.
I20260928 14:42:45.063804 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.66:0
--local_ip_for_outbound_sockets=127.9.33.66
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:44591
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:35605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:45.146783 10115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:45.147018 10115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:45.147045 10115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:45.148351 10115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:45.148401 10115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:45.148432 10115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.66
I20260928 14:42:45.149996 10115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:35605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:44591
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.66
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:45.150215 10115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:45.150418 10115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:45.155663 10121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:45.155648 10120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:45.155647 10123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:45.155982 10115 server_base.cc:1034] running on GCE node
I20260928 14:42:45.156268 10115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:45.156566 10115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:45.157778 10115 hybrid_clock.cc:648] HybridClock initialized: now 1790606565157753 us; error 43 us; skew 500 ppm
Sep 28 14:42:45 dist-test-slave-pv0w krb5kdc[9874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606565, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:45.161082 10115 init.cc:377] Logged in from keytab as kudu/127.9.33.66@KRBTEST.COM (short username kudu)
I20260928 14:42:45.161705 10115 webserver.cc:460] Webserver started at http://127.9.33.66:46321/ using document root <none> and password file <none>
I20260928 14:42:45.161981 10115 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:45.162061 10115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:45.162236 10115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:45.163478 10115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-1/data/instance:
uuid: "9a38af8aa1cc4d9b801acb641d759b26"
format_stamp: "Formatted at 2026-09-28 14:42:45 on dist-test-slave-pv0w"
server_key: "b8cc50731cc0a0bac3f645ea3170bf71"
server_key_iv: "903cb6d72c1791856fa67a3ff6c37b08"
server_key_version: "encryptionkey@0"
I20260928 14:42:45.163942 10115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance:
uuid: "9a38af8aa1cc4d9b801acb641d759b26"
format_stamp: "Formatted at 2026-09-28 14:42:45 on dist-test-slave-pv0w"
server_key: "b8cc50731cc0a0bac3f645ea3170bf71"
server_key_iv: "903cb6d72c1791856fa67a3ff6c37b08"
server_key_version: "encryptionkey@0"
I20260928 14:42:45.165457 10115 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:42:45.166504 10130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:45.166800 10115 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:42:45.166930 10115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-1/wal
uuid: "9a38af8aa1cc4d9b801acb641d759b26"
format_stamp: "Formatted at 2026-09-28 14:42:45 on dist-test-slave-pv0w"
server_key: "b8cc50731cc0a0bac3f645ea3170bf71"
server_key_iv: "903cb6d72c1791856fa67a3ff6c37b08"
server_key_version: "encryptionkey@0"
I20260928 14:42:45.167022 10115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:45.180109 10115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:45.182880 10115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:45.183082 10115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:45.183359 10115 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:42:45.183830 10115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:42:45.183893 10115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:45.183933 10115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:42:45.183964 10115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:45.190716 10115 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.66:41315
I20260928 14:42:45.190759 10243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.66:41315 every 8 connection(s)
I20260928 14:42:45.191145 10115 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
Sep 28 14:42:45 dist-test-slave-pv0w krb5kdc[9874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606565, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:42:45.199105 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 10115
I20260928 14:42:45.199209 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance
I20260928 14:42:45.199440 9349 external_mini_cluster.cc:1428] Setting key 92e67a5936ea8a90e9dc6fc01b5a955b
I20260928 14:42:45.202314 9960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:45.192023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:59341 (local address 127.9.33.126:44591)
0928 14:42:45.192092 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:45.192095 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:45.192335 (+ 240us) server_negotiation.cc:407] Connection header received
0928 14:42:45.193345 (+ 1010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:45.193346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:45.193358 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:45.193493 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:45.194200 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.194825 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.196023 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.196236 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.196302 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:45.198939 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:45.198941 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:45.198942 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:45.198951 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:45.200655 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:45.201132 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:45.201133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:45.201133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:45.201196 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:45.201544 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:45.201546 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:45.201546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:45.201689 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:45.201755 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:45.202133 (+ 378us) server_negotiation.cc:299] Negotiation successful
0928 14:42:45.202219 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260928 14:42:45.202720 10246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:45.192009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:44591 (local address 127.9.33.66:59341)
0928 14:42:45.192227 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:45.192265 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:45.193208 (+ 943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:45.193579 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:45.193583 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:45.193876 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:45.194080 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:45.194085 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.194946 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.194947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:45.195905 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:45.195908 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.196367 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.196368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:45.196502 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:45.197181 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:45.197195 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:45.198809 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:45.200747 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:45.200757 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:45.200760 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:45.201045 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:45.201274 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:45.201275 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:45.201276 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:45.201449 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:45.201765 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:45.201772 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:45.201976 (+ 204us) client_negotiation.cc:769] Sending connection context
0928 14:42:45.202110 (+ 134us) client_negotiation.cc:240] Negotiation successful
0928 14:42:45.202307 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":101,"threads_started":1}
I20260928 14:42:45.203245 10244 heartbeater.cc:347] Connected to a master server at 127.9.33.126:44591
I20260928 14:42:45.203359 10244 heartbeater.cc:464] Registering TS with master...
I20260928 14:42:45.203579 10244 heartbeater.cc:511] Master 127.9.33.126:44591 requested a full tablet report, sending...
I20260928 14:42:45.204044 9919 ts_manager.cc:194] Registered new tserver with Master: 9a38af8aa1cc4d9b801acb641d759b26 (127.9.33.66:41315)
I20260928 14:42:45.204443 9919 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.66@KRBTEST.COM'} at 127.9.33.66:59341
WARNING: no policy specified for kudu/127.9.33.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab.
Entry for principal kudu/127.9.33.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab.
I20260928 14:42:45.228533 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.67:0
--local_ip_for_outbound_sockets=127.9.33.67
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:44591
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:35605
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:45.311980 10249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:45.312203 10249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:45.312229 10249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:45.313356 10249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:45.313392 10249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:45.313454 10249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.67
I20260928 14:42:45.314975 10249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:35605
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:44591
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.67
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:45.315164 10249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:45.315346 10249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:45.320627 10254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:45.320647 10249 server_base.cc:1034] running on GCE node
W20260928 14:42:45.320627 10257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:45.320921 10255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:45.321173 10249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:45.321389 10249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:45.322540 10249 hybrid_clock.cc:648] HybridClock initialized: now 1790606565322526 us; error 29 us; skew 500 ppm
Sep 28 14:42:45 dist-test-slave-pv0w krb5kdc[9874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606565, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:45.325297 10249 init.cc:377] Logged in from keytab as kudu/127.9.33.67@KRBTEST.COM (short username kudu)
I20260928 14:42:45.325771 10249 webserver.cc:460] Webserver started at http://127.9.33.67:35439/ using document root <none> and password file <none>
I20260928 14:42:45.326064 10249 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:45.326138 10249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:45.326290 10249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:45.327422 10249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-2/data/instance:
uuid: "fcc82568618f4bf08ed052011219843c"
format_stamp: "Formatted at 2026-09-28 14:42:45 on dist-test-slave-pv0w"
server_key: "6a4772f9f5b1e5e92b53a9f998cf7964"
server_key_iv: "cea858efb8c5e44e3e0ea26d6b103d2b"
server_key_version: "encryptionkey@0"
I20260928 14:42:45.327872 10249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance:
uuid: "fcc82568618f4bf08ed052011219843c"
format_stamp: "Formatted at 2026-09-28 14:42:45 on dist-test-slave-pv0w"
server_key: "6a4772f9f5b1e5e92b53a9f998cf7964"
server_key_iv: "cea858efb8c5e44e3e0ea26d6b103d2b"
server_key_version: "encryptionkey@0"
I20260928 14:42:45.329280 10249 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:42:45.330104 10264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:45.330307 10249 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260928 14:42:45.330425 10249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-2/wal
uuid: "fcc82568618f4bf08ed052011219843c"
format_stamp: "Formatted at 2026-09-28 14:42:45 on dist-test-slave-pv0w"
server_key: "6a4772f9f5b1e5e92b53a9f998cf7964"
server_key_iv: "cea858efb8c5e44e3e0ea26d6b103d2b"
server_key_version: "encryptionkey@0"
I20260928 14:42:45.330515 10249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:45.348202 10249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:45.350541 10249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:45.350734 10249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:45.351018 10249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:42:45.351356 10249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:42:45.351413 10249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:45.351454 10249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:42:45.351487 10249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:45.357595 10249 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.67:45071
I20260928 14:42:45.357647 10377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.67:45071 every 8 connection(s)
I20260928 14:42:45.358019 10249 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
Sep 28 14:42:45 dist-test-slave-pv0w krb5kdc[9874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606565, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:42:45.364384 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 10249
I20260928 14:42:45.364490 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance
I20260928 14:42:45.364660 9349 external_mini_cluster.cc:1428] Setting key 406d58d3df9bcfc3017983d3b2e5534e
I20260928 14:42:45.367516 9960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:45.358802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:44831 (local address 127.9.33.126:44591)
0928 14:42:45.358885 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:45.358887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:45.359051 (+ 164us) server_negotiation.cc:407] Connection header received
0928 14:42:45.360042 (+ 991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:45.360043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:45.360052 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:45.360109 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:45.360721 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.361226 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.362185 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.362368 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.362420 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:45.364771 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:45.364773 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:45.364773 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:45.364780 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:45.366333 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:45.366716 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:45.366717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:45.366717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:45.366768 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:45.366988 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:45.366989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:45.366989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:45.367088 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:45.367139 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:45.367394 (+ 255us) server_negotiation.cc:299] Negotiation successful
0928 14:42:45.367455 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260928 14:42:45.367784 10380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:45.358754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:44591 (local address 127.9.33.67:44831)
0928 14:42:45.358915 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:45.358959 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:45.359952 (+ 993us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:45.360178 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:45.360182 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:45.360452 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:45.360649 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:45.360653 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.361308 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.361308 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:45.362111 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:45.362113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.362446 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.362447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:45.362555 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:45.363142 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:45.363151 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:45.364675 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:45.366416 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:45.366423 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:45.366425 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:45.366649 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:45.366826 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:45.366827 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:45.366827 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:45.366930 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:45.367143 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:45.367147 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:45.367282 (+ 135us) client_negotiation.cc:769] Sending connection context
0928 14:42:45.367367 (+ 85us) client_negotiation.cc:240] Negotiation successful
0928 14:42:45.367488 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":95,"threads_started":1}
I20260928 14:42:45.368233 10378 heartbeater.cc:347] Connected to a master server at 127.9.33.126:44591
I20260928 14:42:45.368340 10378 heartbeater.cc:464] Registering TS with master...
I20260928 14:42:45.368515 10378 heartbeater.cc:511] Master 127.9.33.126:44591 requested a full tablet report, sending...
I20260928 14:42:45.368873 9919 ts_manager.cc:194] Registered new tserver with Master: fcc82568618f4bf08ed052011219843c (127.9.33.67:45071)
I20260928 14:42:45.369257 9919 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.67@KRBTEST.COM'} at 127.9.33.67:44831
I20260928 14:42:45.376631 9349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:42:45.384413 9960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:45.378414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39610 (local address 127.9.33.126:44591)
0928 14:42:45.378471 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:45.378473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:45.378482 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:42:45.378506 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:45.378507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:45.378523 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:45.378576 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:45.379165 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.379638 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.380438 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.380630 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.380686 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:45.381996 (+ 1310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:45.381997 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:45.381998 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:45.382003 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:45.383287 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:45.383612 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:45.383613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:45.383613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:45.383664 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:45.383931 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:45.383931 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:45.383932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:45.384077 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:45.384155 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:45.384286 (+ 131us) server_negotiation.cc:299] Negotiation successful
0928 14:42:45.384347 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260928 14:42:45.386725 9919 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39610:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:42:45.388482 9919 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:42:45.395545 10392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:45.391289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:41315 (local address 127.0.0.1:48054)
0928 14:42:45.391710 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:45.391757 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:45.391810 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:45.392064 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:45.392065 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:45.392068 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:42:45.392107 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:45.392108 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.393418 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.393419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:45.394473 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:45.394476 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.395305 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.395306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:45.395375 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:45.395386 (+ 11us) client_negotiation.cc:769] Sending connection context
0928 14:42:45.395446 (+ 60us) client_negotiation.cc:240] Negotiation successful
0928 14:42:45.395484 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":67,"threads_started":1}
I20260928 14:42:45.395612 10394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:45.391502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48054 (local address 127.9.33.66:41315)
0928 14:42:45.391670 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:45.391673 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:45.391908 (+ 235us) server_negotiation.cc:407] Connection header received
0928 14:42:45.391952 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:45.391954 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:45.391971 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:45.392049 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:42:45.392179 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.393304 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.394561 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.395222 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.395297 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:45.395348 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:45.395490 (+ 142us) server_negotiation.cc:299] Negotiation successful
0928 14:42:45.395531 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":67,"threads_started":1}
I20260928 14:42:45.396100 10390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:45.391079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:45071 (local address 127.0.0.1:54804)
0928 14:42:45.391227 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:45.391271 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:45.391352 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:45.391593 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:45.391596 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:45.391599 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:42:45.391761 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:45.391764 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.392756 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.392757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:45.394034 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:45.394037 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.395898 (+ 1861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.395899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:45.395972 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:45.395975 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:42:45.396022 (+ 47us) client_negotiation.cc:240] Negotiation successful
0928 14:42:45.396053 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":1,"thread_start_us":67,"threads_started":1}
I20260928 14:42:45.396165 10391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:45.391205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54804 (local address 127.9.33.67:45071)
0928 14:42:45.391421 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:45.391426 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:45.391436 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:42:45.391474 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:45.391476 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:45.391493 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:45.391588 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:42:45.391860 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.392649 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.394113 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.394853 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.394931 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:45.394991 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:45.396044 (+ 1053us) server_negotiation.cc:299] Negotiation successful
0928 14:42:45.396089 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":102,"threads_started":1}
I20260928 14:42:45.396513 10389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:45.391079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:34369 (local address 127.0.0.1:48188)
0928 14:42:45.391473 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:45.391484 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:45.391538 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:45.392399 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:45.392400 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:45.392402 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:42:45.392451 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:45.392452 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.393674 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.393675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:45.394850 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:45.394854 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.396266 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.396267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:45.396338 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:45.396342 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:42:45.396406 (+ 64us) client_negotiation.cc:240] Negotiation successful
0928 14:42:45.396443 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"mutex_wait_us":2,"thread_start_us":84,"threads_started":1}
I20260928 14:42:45.396559 10393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:45.391277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48188 (local address 127.9.33.65:34369)
0928 14:42:45.391849 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:45.391854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:45.391864 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:42:45.392216 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:45.392219 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:45.392234 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:45.392303 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:42:45.392524 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.393571 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.394937 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.395707 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.395805 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:45.395862 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:45.396437 (+ 575us) server_negotiation.cc:299] Negotiation successful
0928 14:42:45.396484 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":107,"threads_started":1}
I20260928 14:42:45.396432 10178 tablet_service.cc:1467] Processing CreateTablet for tablet 4cb54062073a4fdbaf24b1a800ab0cff (DEFAULT_TABLE table=test-table [id=da3056af79264657bfbf57352f7a1b6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:42:45.396833 10178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cb54062073a4fdbaf24b1a800ab0cff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:45.396860 10312 tablet_service.cc:1467] Processing CreateTablet for tablet 4cb54062073a4fdbaf24b1a800ab0cff (DEFAULT_TABLE table=test-table [id=da3056af79264657bfbf57352f7a1b6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:42:45.397115 10312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cb54062073a4fdbaf24b1a800ab0cff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:45.399303 10395 tablet_bootstrap.cc:492] T 4cb54062073a4fdbaf24b1a800ab0cff P 9a38af8aa1cc4d9b801acb641d759b26: Bootstrap starting.
I20260928 14:42:45.400023 10395 tablet_bootstrap.cc:654] T 4cb54062073a4fdbaf24b1a800ab0cff P 9a38af8aa1cc4d9b801acb641d759b26: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:45.400005 10396 tablet_bootstrap.cc:492] T 4cb54062073a4fdbaf24b1a800ab0cff P fcc82568618f4bf08ed052011219843c: Bootstrap starting.
I20260928 14:42:45.400352 10044 tablet_service.cc:1467] Processing CreateTablet for tablet 4cb54062073a4fdbaf24b1a800ab0cff (DEFAULT_TABLE table=test-table [id=da3056af79264657bfbf57352f7a1b6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:42:45.400609 10395 log.cc:824] T 4cb54062073a4fdbaf24b1a800ab0cff P 9a38af8aa1cc4d9b801acb641d759b26: Log is configured to *not* fsync() on all Append() calls
I20260928 14:42:45.400704 10396 tablet_bootstrap.cc:654] T 4cb54062073a4fdbaf24b1a800ab0cff P fcc82568618f4bf08ed052011219843c: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:45.400703 10044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cb54062073a4fdbaf24b1a800ab0cff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:45.401147 10396 log.cc:824] T 4cb54062073a4fdbaf24b1a800ab0cff P fcc82568618f4bf08ed052011219843c: Log is configured to *not* fsync() on all Append() calls
I20260928 14:42:45.401847 10396 tablet_bootstrap.cc:492] T 4cb54062073a4fdbaf24b1a800ab0cff P fcc82568618f4bf08ed052011219843c: No bootstrap required, opened a new log
I20260928 14:42:45.401934 10396 ts_tablet_manager.cc:1397] T 4cb54062073a4fdbaf24b1a800ab0cff P fcc82568618f4bf08ed052011219843c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:42:45.403417 10399 tablet_bootstrap.cc:492] T 4cb54062073a4fdbaf24b1a800ab0cff P 04607244c7bd4521b011e1021cfc3912: Bootstrap starting.
I20260928 14:42:45.403477 10396 raft_consensus.cc:348] T 4cb54062073a4fdbaf24b1a800ab0cff P fcc82568618f4bf08ed052011219843c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04607244c7bd4521b011e1021cfc3912" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 34369 } } peers { permanent_uuid: "9a38af8aa1cc4d9b801acb641d759b26" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 41315 } } peers { permanent_uuid: "fcc82568618f4bf08ed052011219843c" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45071 } }
I20260928 14:42:45.403707 10396 raft_consensus.cc:374] T 4cb54062073a4fdbaf24b1a800ab0cff P fcc82568618f4bf08ed052011219843c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:45.403785 10396 raft_consensus.cc:729] T 4cb54062073a4fdbaf24b1a800ab0cff P fcc82568618f4bf08ed052011219843c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcc82568618f4bf08ed052011219843c, State: Initialized, Role: FOLLOWER
I20260928 14:42:45.403900 10396 consensus_queue.cc:260] T 4cb54062073a4fdbaf24b1a800ab0cff P fcc82568618f4bf08ed052011219843c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04607244c7bd4521b011e1021cfc3912" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 34369 } } peers { permanent_uuid: "9a38af8aa1cc4d9b801acb641d759b26" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 41315 } } peers { permanent_uuid: "fcc82568618f4bf08ed052011219843c" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45071 } }
I20260928 14:42:45.404095 10399 tablet_bootstrap.cc:654] T 4cb54062073a4fdbaf24b1a800ab0cff P 04607244c7bd4521b011e1021cfc3912: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:45.404158 10395 tablet_bootstrap.cc:492] T 4cb54062073a4fdbaf24b1a800ab0cff P 9a38af8aa1cc4d9b801acb641d759b26: No bootstrap required, opened a new log
I20260928 14:42:45.404201 10396 ts_tablet_manager.cc:1428] T 4cb54062073a4fdbaf24b1a800ab0cff P fcc82568618f4bf08ed052011219843c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260928 14:42:45.404233 10395 ts_tablet_manager.cc:1397] T 4cb54062073a4fdbaf24b1a800ab0cff P 9a38af8aa1cc4d9b801acb641d759b26: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260928 14:42:45.404368 10378 heartbeater.cc:503] Master 127.9.33.126:44591 was elected leader, sending a full tablet report...
I20260928 14:42:45.404660 10399 log.cc:824] T 4cb54062073a4fdbaf24b1a800ab0cff P 04607244c7bd4521b011e1021cfc3912: Log is configured to *not* fsync() on all Append() calls
I20260928 14:42:45.405463 10399 tablet_bootstrap.cc:492] T 4cb54062073a4fdbaf24b1a800ab0cff P 04607244c7bd4521b011e1021cfc3912: No bootstrap required, opened a new log
I20260928 14:42:45.405550 10399 ts_tablet_manager.cc:1397] T 4cb54062073a4fdbaf24b1a800ab0cff P 04607244c7bd4521b011e1021cfc3912: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:42:45.406178 10395 raft_consensus.cc:348] T 4cb54062073a4fdbaf24b1a800ab0cff P 9a38af8aa1cc4d9b801acb641d759b26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04607244c7bd4521b011e1021cfc3912" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 34369 } } peers { permanent_uuid: "9a38af8aa1cc4d9b801acb641d759b26" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 41315 } } peers { permanent_uuid: "fcc82568618f4bf08ed052011219843c" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45071 } }
I20260928 14:42:45.406318 10395 raft_consensus.cc:374] T 4cb54062073a4fdbaf24b1a800ab0cff P 9a38af8aa1cc4d9b801acb641d759b26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:45.406347 10395 raft_consensus.cc:729] T 4cb54062073a4fdbaf24b1a800ab0cff P 9a38af8aa1cc4d9b801acb641d759b26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a38af8aa1cc4d9b801acb641d759b26, State: Initialized, Role: FOLLOWER
I20260928 14:42:45.406477 10395 consensus_queue.cc:260] T 4cb54062073a4fdbaf24b1a800ab0cff P 9a38af8aa1cc4d9b801acb641d759b26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04607244c7bd4521b011e1021cfc3912" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 34369 } } peers { permanent_uuid: "9a38af8aa1cc4d9b801acb641d759b26" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 41315 } } peers { permanent_uuid: "fcc82568618f4bf08ed052011219843c" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45071 } }
I20260928 14:42:45.406754 10244 heartbeater.cc:503] Master 127.9.33.126:44591 was elected leader, sending a full tablet report...
I20260928 14:42:45.406761 10395 ts_tablet_manager.cc:1428] T 4cb54062073a4fdbaf24b1a800ab0cff P 9a38af8aa1cc4d9b801acb641d759b26: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260928 14:42:45.407266 10399 raft_consensus.cc:348] T 4cb54062073a4fdbaf24b1a800ab0cff P 04607244c7bd4521b011e1021cfc3912 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04607244c7bd4521b011e1021cfc3912" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 34369 } } peers { permanent_uuid: "9a38af8aa1cc4d9b801acb641d759b26" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 41315 } } peers { permanent_uuid: "fcc82568618f4bf08ed052011219843c" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45071 } }
I20260928 14:42:45.407457 10399 raft_consensus.cc:374] T 4cb54062073a4fdbaf24b1a800ab0cff P 04607244c7bd4521b011e1021cfc3912 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:45.407505 10399 raft_consensus.cc:729] T 4cb54062073a4fdbaf24b1a800ab0cff P 04607244c7bd4521b011e1021cfc3912 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04607244c7bd4521b011e1021cfc3912, State: Initialized, Role: FOLLOWER
I20260928 14:42:45.407620 10399 consensus_queue.cc:260] T 4cb54062073a4fdbaf24b1a800ab0cff P 04607244c7bd4521b011e1021cfc3912 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04607244c7bd4521b011e1021cfc3912" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 34369 } } peers { permanent_uuid: "9a38af8aa1cc4d9b801acb641d759b26" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 41315 } } peers { permanent_uuid: "fcc82568618f4bf08ed052011219843c" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45071 } }
I20260928 14:42:45.407918 10399 ts_tablet_manager.cc:1428] T 4cb54062073a4fdbaf24b1a800ab0cff P 04607244c7bd4521b011e1021cfc3912: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260928 14:42:45.407976 10110 heartbeater.cc:503] Master 127.9.33.126:44591 was elected leader, sending a full tablet report...
W20260928 14:42:45.441845 10245 tablet.cc:2369] T 4cb54062073a4fdbaf24b1a800ab0cff P 9a38af8aa1cc4d9b801acb641d759b26: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:42:45.500288 10403 raft_consensus.cc:482] T 4cb54062073a4fdbaf24b1a800ab0cff P 04607244c7bd4521b011e1021cfc3912 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:42:45.500440 10403 raft_consensus.cc:504] T 4cb54062073a4fdbaf24b1a800ab0cff P 04607244c7bd4521b011e1021cfc3912 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04607244c7bd4521b011e1021cfc3912" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 34369 } } peers { permanent_uuid: "9a38af8aa1cc4d9b801acb641d759b26" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 41315 } } peers { permanent_uuid: "fcc82568618f4bf08ed052011219843c" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45071 } }
I20260928 14:42:45.500790 10403 leader_election.cc:288] T 4cb54062073a4fdbaf24b1a800ab0cff P 04607244c7bd4521b011e1021cfc3912 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a38af8aa1cc4d9b801acb641d759b26 (127.9.33.66:41315), fcc82568618f4bf08ed052011219843c (127.9.33.67:45071)
I20260928 14:42:45.503752 10391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:45.501140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:32977 (local address 127.9.33.67:45071)
0928 14:42:45.501213 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:45.501220 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:45.501229 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:42:45.501254 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:45.501255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:45.501263 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:45.501347 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:42:45.501531 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.502016 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.502990 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.503377 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.503462 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:45.503473 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:45.503626 (+ 153us) server_negotiation.cc:299] Negotiation successful
0928 14:42:45.503666 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260928 14:42:45.503765 10112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:45.501025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:45071 (local address 127.9.33.65:32977)
0928 14:42:45.501069 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:45.501082 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:45.501129 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:45.501372 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:45.501373 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:45.501376 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:42:45.501451 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:45.501453 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.502113 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.502114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:45.502891 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:45.502893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.503460 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.503461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:45.503546 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:45.503550 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:42:45.503619 (+ 69us) client_negotiation.cc:240] Negotiation successful
0928 14:42:45.503660 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260928 14:42:45.504055 10394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:45.501145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:33551 (local address 127.9.33.66:41315)
0928 14:42:45.501195 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:45.501198 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:45.501452 (+ 254us) server_negotiation.cc:407] Connection header received
0928 14:42:45.501522 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:45.501523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:45.501538 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:45.501611 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:42:45.501776 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.502254 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.503198 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.503590 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.503677 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:45.503687 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:45.503917 (+ 230us) server_negotiation.cc:299] Negotiation successful
0928 14:42:45.503979 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260928 14:42:45.504103 10332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cb54062073a4fdbaf24b1a800ab0cff" candidate_uuid: "04607244c7bd4521b011e1021cfc3912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcc82568618f4bf08ed052011219843c" is_pre_election: true
I20260928 14:42:45.504276 10332 raft_consensus.cc:2454] T 4cb54062073a4fdbaf24b1a800ab0cff P fcc82568618f4bf08ed052011219843c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04607244c7bd4521b011e1021cfc3912 in term 0.
I20260928 14:42:45.504292 10404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:45.501025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:41315 (local address 127.9.33.65:33551)
0928 14:42:45.501367 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:45.501381 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:45.501454 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:45.501614 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:45.501615 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:45.501616 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:42:45.501700 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:45.501702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.502341 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.502342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:45.503092 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:45.503094 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.503685 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.503686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:45.503800 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:45.503804 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:42:45.504168 (+ 364us) client_negotiation.cc:240] Negotiation successful
0928 14:42:45.504219 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":186,"thread_start_us":109,"threads_started":1}
I20260928 14:42:45.504530 9999 leader_election.cc:302] T 4cb54062073a4fdbaf24b1a800ab0cff P 04607244c7bd4521b011e1021cfc3912 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04607244c7bd4521b011e1021cfc3912, fcc82568618f4bf08ed052011219843c; no voters:
I20260928 14:42:45.504694 10403 raft_consensus.cc:2781] T 4cb54062073a4fdbaf24b1a800ab0cff P 04607244c7bd4521b011e1021cfc3912 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:42:45.504662 10198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cb54062073a4fdbaf24b1a800ab0cff" candidate_uuid: "04607244c7bd4521b011e1021cfc3912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a38af8aa1cc4d9b801acb641d759b26" is_pre_election: true
I20260928 14:42:45.504747 10403 raft_consensus.cc:482] T 4cb54062073a4fdbaf24b1a800ab0cff P 04607244c7bd4521b011e1021cfc3912 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:42:45.504771 10403 raft_consensus.cc:3037] T 4cb54062073a4fdbaf24b1a800ab0cff P 04607244c7bd4521b011e1021cfc3912 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:45.504817 10198 raft_consensus.cc:2454] T 4cb54062073a4fdbaf24b1a800ab0cff P 9a38af8aa1cc4d9b801acb641d759b26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04607244c7bd4521b011e1021cfc3912 in term 0.
I20260928 14:42:45.505491 10403 raft_consensus.cc:504] T 4cb54062073a4fdbaf24b1a800ab0cff P 04607244c7bd4521b011e1021cfc3912 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04607244c7bd4521b011e1021cfc3912" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 34369 } } peers { permanent_uuid: "9a38af8aa1cc4d9b801acb641d759b26" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 41315 } } peers { permanent_uuid: "fcc82568618f4bf08ed052011219843c" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45071 } }
I20260928 14:42:45.505640 10403 leader_election.cc:288] T 4cb54062073a4fdbaf24b1a800ab0cff P 04607244c7bd4521b011e1021cfc3912 [CANDIDATE]: Term 1 election: Requested vote from peers 9a38af8aa1cc4d9b801acb641d759b26 (127.9.33.66:41315), fcc82568618f4bf08ed052011219843c (127.9.33.67:45071)
I20260928 14:42:45.505822 10332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cb54062073a4fdbaf24b1a800ab0cff" candidate_uuid: "04607244c7bd4521b011e1021cfc3912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcc82568618f4bf08ed052011219843c"
I20260928 14:42:45.505825 10198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cb54062073a4fdbaf24b1a800ab0cff" candidate_uuid: "04607244c7bd4521b011e1021cfc3912" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a38af8aa1cc4d9b801acb641d759b26"
I20260928 14:42:45.505901 10332 raft_consensus.cc:3037] T 4cb54062073a4fdbaf24b1a800ab0cff P fcc82568618f4bf08ed052011219843c [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:45.505903 10198 raft_consensus.cc:3037] T 4cb54062073a4fdbaf24b1a800ab0cff P 9a38af8aa1cc4d9b801acb641d759b26 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:45.506775 10332 raft_consensus.cc:2454] T 4cb54062073a4fdbaf24b1a800ab0cff P fcc82568618f4bf08ed052011219843c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04607244c7bd4521b011e1021cfc3912 in term 1.
I20260928 14:42:45.506775 10198 raft_consensus.cc:2454] T 4cb54062073a4fdbaf24b1a800ab0cff P 9a38af8aa1cc4d9b801acb641d759b26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04607244c7bd4521b011e1021cfc3912 in term 1.
I20260928 14:42:45.506943 9999 leader_election.cc:302] T 4cb54062073a4fdbaf24b1a800ab0cff P 04607244c7bd4521b011e1021cfc3912 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04607244c7bd4521b011e1021cfc3912, fcc82568618f4bf08ed052011219843c; no voters:
I20260928 14:42:45.507037 10403 raft_consensus.cc:2781] T 4cb54062073a4fdbaf24b1a800ab0cff P 04607244c7bd4521b011e1021cfc3912 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:42:45.507225 10403 raft_consensus.cc:686] T 4cb54062073a4fdbaf24b1a800ab0cff P 04607244c7bd4521b011e1021cfc3912 [term 1 LEADER]: Becoming Leader. State: Replica: 04607244c7bd4521b011e1021cfc3912, State: Running, Role: LEADER
I20260928 14:42:45.507342 10403 consensus_queue.cc:237] T 4cb54062073a4fdbaf24b1a800ab0cff P 04607244c7bd4521b011e1021cfc3912 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04607244c7bd4521b011e1021cfc3912" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 34369 } } peers { permanent_uuid: "9a38af8aa1cc4d9b801acb641d759b26" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 41315 } } peers { permanent_uuid: "fcc82568618f4bf08ed052011219843c" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45071 } }
I20260928 14:42:45.508351 9919 catalog_manager.cc:5495] T 4cb54062073a4fdbaf24b1a800ab0cff P 04607244c7bd4521b011e1021cfc3912 reported cstate change: term changed from 0 to 1, leader changed from <none> to 04607244c7bd4521b011e1021cfc3912 (127.9.33.65). New cstate: current_term: 1 leader_uuid: "04607244c7bd4521b011e1021cfc3912" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04607244c7bd4521b011e1021cfc3912" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 34369 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a38af8aa1cc4d9b801acb641d759b26" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 41315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcc82568618f4bf08ed052011219843c" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45071 } health_report { overall_health: UNKNOWN } } }
Sep 28 14:42:45 dist-test-slave-pv0w krb5kdc[9874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606564, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.65@KRBTEST.COM
I20260928 14:42:45.519310 10393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:45.511202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48198 (local address 127.9.33.65:34369)
0928 14:42:45.511302 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:45.511304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:45.511424 (+ 120us) server_negotiation.cc:407] Connection header received
0928 14:42:45.511496 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:45.511496 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:45.511509 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:45.511582 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:45.512372 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.512859 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.513618 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.513819 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.513898 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:45.516415 (+ 2517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:45.516426 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:45.516426 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:45.516434 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:45.518039 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:45.518450 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:45.518454 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:45.518455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:45.518519 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:45.518726 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:45.518727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:45.518727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:45.518880 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:45.519078 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:45.519116 (+ 38us) server_negotiation.cc:299] Negotiation successful
0928 14:42:45.519197 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260928 14:42:45.524673 10111 tablet.cc:2369] T 4cb54062073a4fdbaf24b1a800ab0cff P 04607244c7bd4521b011e1021cfc3912: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 28 14:42:45 dist-test-slave-pv0w krb5kdc[9874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606565, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 28 14:42:45 dist-test-slave-pv0w krb5kdc[9874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606565, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.33.65@KRBTEST.COM
I20260928 14:42:45.544476 10393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:45.536541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48210 (local address 127.9.33.65:34369)
0928 14:42:45.536624 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:45.536627 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:45.536727 (+ 100us) server_negotiation.cc:407] Connection header received
0928 14:42:45.536869 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:45.536870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:45.536884 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:45.536953 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:45.537537 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.538093 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.538852 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.539097 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.539191 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:45.541554 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:45.541556 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:45.541556 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:45.541565 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:45.542908 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:45.543319 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:45.543320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:45.543321 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:45.543388 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:45.543615 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:45.543616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:45.543617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:45.543816 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:45.543925 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:45.544254 (+ 329us) server_negotiation.cc:299] Negotiation successful
0928 14:42:45.544357 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Sep 28 14:42:45 dist-test-slave-pv0w krb5kdc[9874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 14:42:45 dist-test-slave-pv0w krb5kdc[9874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.65@KRBTEST.COM
I20260928 14:42:45.568241 10393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:45.560936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48214 (local address 127.9.33.65:34369)
0928 14:42:45.561001 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:45.561003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:45.561084 (+ 81us) server_negotiation.cc:407] Connection header received
0928 14:42:45.561134 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:45.561135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:45.561144 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:45.561220 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:45.561782 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.562274 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.563049 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:45.563248 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:45.563314 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:45.565691 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:45.565693 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:45.565694 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:45.565702 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:45.567110 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:45.567454 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:45.567456 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:45.567456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:45.567514 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:45.567694 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:45.567694 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:45.567695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:45.567875 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:45.567966 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:45.568055 (+ 89us) server_negotiation.cc:299] Negotiation successful
0928 14:42:45.568117 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260928 14:42:45.568799 10024 tablet_service.cc:2908] Rejecting scan request for tablet 4cb54062073a4fdbaf24b1a800ab0cff: Uninitialized: clean time has not yet been initialized
I20260928 14:42:45.569669 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 9981
I20260928 14:42:45.574803 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 10115
I20260928 14:42:45.579788 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 10249
I20260928 14:42:45.584211 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 9887
2026-09-28T14:42:45Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2061 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:42:45 dist-test-slave-pv0w krb5kdc[10419](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:42:45 dist-test-slave-pv0w krb5kdc[10419](info): set up 2 sockets
Sep 28 14:42:45 dist-test-slave-pv0w krb5kdc[10419](info): commencing operation
krb5kdc: starting...
W20260928 14:42:46.632604 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:42:46 dist-test-slave-pv0w krb5kdc[10419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:42:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:42:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.33.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
I20260928 14:42:46.759447 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:37901
--webserver_interface=127.9.33.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:45333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:37901
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:46.841670 10432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:46.841952 10432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:46.841991 10432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:46.843235 10432 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:42:46.843313 10432 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260928 14:42:46.843338 10432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:46.843358 10432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:46.843391 10432 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:42:46.843420 10432 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:42:46.844998 10432 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:45333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:37901
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:37901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:46.845299 10432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:46.845546 10432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:46.850955 10440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:46.850961 10438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:46.850979 10432 server_base.cc:1034] running on GCE node
W20260928 14:42:46.850961 10437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:46.851462 10432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:46.851786 10432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:46.852941 10432 hybrid_clock.cc:648] HybridClock initialized: now 1790606566852920 us; error 41 us; skew 500 ppm
Sep 28 14:42:46 dist-test-slave-pv0w krb5kdc[10419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606566, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:46.862820 10432 init.cc:377] Logged in from keytab as kudu/127.9.33.126@KRBTEST.COM (short username kudu)
I20260928 14:42:46.863304 10432 webserver.cc:460] Webserver started at http://127.9.33.126:38633/ using document root <none> and password file <none>
I20260928 14:42:46.863552 10432 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:46.863600 10432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:46.863751 10432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:46.864898 10432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "990c5809ef95459b882bfeb2232d16b5"
format_stamp: "Formatted at 2026-09-28 14:42:46 on dist-test-slave-pv0w"
server_key: "46baae61fce0ebd2c8eb954415570261"
server_key_iv: "e850c124938d268ef9ff8243d3a8b351"
server_key_version: "encryptionkey@0"
I20260928 14:42:46.865279 10432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "990c5809ef95459b882bfeb2232d16b5"
format_stamp: "Formatted at 2026-09-28 14:42:46 on dist-test-slave-pv0w"
server_key: "46baae61fce0ebd2c8eb954415570261"
server_key_iv: "e850c124938d268ef9ff8243d3a8b351"
server_key_version: "encryptionkey@0"
I20260928 14:42:46.866679 10432 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260928 14:42:46.867466 10447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:46.867682 10432 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:42:46.867808 10432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "990c5809ef95459b882bfeb2232d16b5"
format_stamp: "Formatted at 2026-09-28 14:42:46 on dist-test-slave-pv0w"
server_key: "46baae61fce0ebd2c8eb954415570261"
server_key_iv: "e850c124938d268ef9ff8243d3a8b351"
server_key_version: "encryptionkey@0"
I20260928 14:42:46.867892 10432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:46.892107 10432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:46.894572 10432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:46.894773 10432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:46.899499 10432 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:37901
I20260928 14:42:46.899609 10509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:37901 every 8 connection(s)
I20260928 14:42:46.900004 10432 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:42:46.900738 10510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:46.903061 10510 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 990c5809ef95459b882bfeb2232d16b5: Bootstrap starting.
I20260928 14:42:46.903673 10510 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 990c5809ef95459b882bfeb2232d16b5: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:46.903950 10510 log.cc:824] T 00000000000000000000000000000000 P 990c5809ef95459b882bfeb2232d16b5: Log is configured to *not* fsync() on all Append() calls
I20260928 14:42:46.904677 10510 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 990c5809ef95459b882bfeb2232d16b5: No bootstrap required, opened a new log
I20260928 14:42:46.905122 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 10432
I20260928 14:42:46.905220 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:42:46.905378 9349 external_mini_cluster.cc:1428] Setting key 6c90844bd6cac1f8e2c1bf6e3f7d284b
I20260928 14:42:46.907683 10510 raft_consensus.cc:348] T 00000000000000000000000000000000 P 990c5809ef95459b882bfeb2232d16b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "990c5809ef95459b882bfeb2232d16b5" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37901 } }
I20260928 14:42:46.907919 10510 raft_consensus.cc:374] T 00000000000000000000000000000000 P 990c5809ef95459b882bfeb2232d16b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:46.907963 10510 raft_consensus.cc:729] T 00000000000000000000000000000000 P 990c5809ef95459b882bfeb2232d16b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 990c5809ef95459b882bfeb2232d16b5, State: Initialized, Role: FOLLOWER
I20260928 14:42:46.908102 10510 consensus_queue.cc:260] T 00000000000000000000000000000000 P 990c5809ef95459b882bfeb2232d16b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "990c5809ef95459b882bfeb2232d16b5" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37901 } }
I20260928 14:42:46.908206 10510 raft_consensus.cc:388] T 00000000000000000000000000000000 P 990c5809ef95459b882bfeb2232d16b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:42:46.908244 10510 raft_consensus.cc:482] T 00000000000000000000000000000000 P 990c5809ef95459b882bfeb2232d16b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:42:46.908301 10510 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 990c5809ef95459b882bfeb2232d16b5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:46.908898 10510 raft_consensus.cc:504] T 00000000000000000000000000000000 P 990c5809ef95459b882bfeb2232d16b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "990c5809ef95459b882bfeb2232d16b5" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37901 } }
I20260928 14:42:46.909024 10510 leader_election.cc:302] T 00000000000000000000000000000000 P 990c5809ef95459b882bfeb2232d16b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 990c5809ef95459b882bfeb2232d16b5; no voters:
I20260928 14:42:46.909227 10510 leader_election.cc:288] T 00000000000000000000000000000000 P 990c5809ef95459b882bfeb2232d16b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:42:46.909252 10515 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 990c5809ef95459b882bfeb2232d16b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:42:46.909372 10515 raft_consensus.cc:686] T 00000000000000000000000000000000 P 990c5809ef95459b882bfeb2232d16b5 [term 1 LEADER]: Becoming Leader. State: Replica: 990c5809ef95459b882bfeb2232d16b5, State: Running, Role: LEADER
I20260928 14:42:46.909520 10510 sys_catalog.cc:564] T 00000000000000000000000000000000 P 990c5809ef95459b882bfeb2232d16b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:42:46.909515 10515 consensus_queue.cc:237] T 00000000000000000000000000000000 P 990c5809ef95459b882bfeb2232d16b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "990c5809ef95459b882bfeb2232d16b5" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37901 } }
I20260928 14:42:46.910020 10517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 990c5809ef95459b882bfeb2232d16b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 990c5809ef95459b882bfeb2232d16b5. Latest consensus state: current_term: 1 leader_uuid: "990c5809ef95459b882bfeb2232d16b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "990c5809ef95459b882bfeb2232d16b5" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37901 } } }
I20260928 14:42:46.910116 10517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 990c5809ef95459b882bfeb2232d16b5 [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:46.910352 10516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 990c5809ef95459b882bfeb2232d16b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "990c5809ef95459b882bfeb2232d16b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "990c5809ef95459b882bfeb2232d16b5" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37901 } } }
I20260928 14:42:46.910686 10516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 990c5809ef95459b882bfeb2232d16b5 [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:46.910923 10523 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:42:46.911666 10523 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 28 14:42:46 dist-test-slave-pv0w krb5kdc[10419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:42:46.913517 10523 catalog_manager.cc:1347] Generated new cluster ID: 93cc5933df4642dbb911be5ac4645c6a
I20260928 14:42:46.913573 10523 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:42:46.917320 10514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:46.905978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49196 (local address 127.9.33.126:37901)
0928 14:42:46.906175 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:46.906185 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:46.906211 (+ 26us) server_negotiation.cc:407] Connection header received
0928 14:42:46.907175 (+ 964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:46.907185 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:46.907245 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:46.907363 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:46.908182 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:46.909262 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:46.910298 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:46.911078 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:46.911230 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:46.913939 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:46.913944 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:46.913944 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:46.913956 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:46.915680 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:46.916137 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:46.916140 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:46.916141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:46.916237 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:46.916456 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:46.916457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:46.916457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:46.916706 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:46.916897 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:46.916975 (+ 78us) server_negotiation.cc:299] Negotiation successful
0928 14:42:46.917118 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":104,"threads_started":1}
I20260928 14:42:46.925069 10523 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:42:46.925810 10523 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:42:46.944144 10523 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 990c5809ef95459b882bfeb2232d16b5: Generated new TSK 0
I20260928 14:42:46.944406 10523 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.33.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab.
Entry for principal kudu/127.9.33.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab.
I20260928 14:42:46.971951 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.65:0
--local_ip_for_outbound_sockets=127.9.33.65
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:37901
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:45333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:47.056082 10536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:47.056353 10536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:47.056391 10536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:47.057550 10536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:47.057587 10536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:47.057607 10536 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 14:42:47.057632 10536 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.65
I20260928 14:42:47.059031 10536 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:45333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.33.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.33.126:37901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10536
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.65
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:47.059221 10536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:47.059412 10536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:47.064615 10542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:47.064608 10541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:47.064579 10544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:47.065948 10536 server_base.cc:1034] running on GCE node
I20260928 14:42:47.066156 10536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:47.066355 10536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:47.067539 10536 hybrid_clock.cc:648] HybridClock initialized: now 1790606567067521 us; error 37 us; skew 500 ppm
Sep 28 14:42:47 dist-test-slave-pv0w krb5kdc[10419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606567, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:47.070062 10536 init.cc:377] Logged in from keytab as kudu/127.9.33.65@KRBTEST.COM (short username kudu)
I20260928 14:42:47.070580 10536 webserver.cc:460] Webserver started at http://127.9.33.65:37573/ using document root <none> and password file <none>
I20260928 14:42:47.070873 10536 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:47.070945 10536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:47.071096 10536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:47.072284 10536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-0/data/instance:
uuid: "887d7faee69d4abea170536a8cdde4ce"
format_stamp: "Formatted at 2026-09-28 14:42:47 on dist-test-slave-pv0w"
server_key: "7a3d4b09e31b5df765ebf8b6cac65e9e"
server_key_iv: "cd6a4938cf7d379c5427eaea6beda78f"
server_key_version: "encryptionkey@0"
I20260928 14:42:47.072678 10536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance:
uuid: "887d7faee69d4abea170536a8cdde4ce"
format_stamp: "Formatted at 2026-09-28 14:42:47 on dist-test-slave-pv0w"
server_key: "7a3d4b09e31b5df765ebf8b6cac65e9e"
server_key_iv: "cd6a4938cf7d379c5427eaea6beda78f"
server_key_version: "encryptionkey@0"
I20260928 14:42:47.074046 10536 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:42:47.074805 10551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:47.075004 10536 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:42:47.075114 10536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-0/wal
uuid: "887d7faee69d4abea170536a8cdde4ce"
format_stamp: "Formatted at 2026-09-28 14:42:47 on dist-test-slave-pv0w"
server_key: "7a3d4b09e31b5df765ebf8b6cac65e9e"
server_key_iv: "cd6a4938cf7d379c5427eaea6beda78f"
server_key_version: "encryptionkey@0"
I20260928 14:42:47.075189 10536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:47.085150 10536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:47.087004 10536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:47.087159 10536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:47.087705 10536 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:42:47.087819 10536 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:47.087863 10536 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:42:47.087891 10536 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 14:42:47 dist-test-slave-pv0w krb5kdc[10419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606567, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:42:47.095165 10536 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.65:37863
I20260928 14:42:47.095238 10666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.65:37863 every 8 connection(s)
I20260928 14:42:47.095585 10536 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
I20260928 14:42:47.096961 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 10536
I20260928 14:42:47.097072 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance
I20260928 14:42:47.097239 9349 external_mini_cluster.cc:1428] Setting key 50176123c93177dd4fc1d29ce0ec74b4
I20260928 14:42:47.097689 10514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.088177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:40645 (local address 127.9.33.126:37901)
0928 14:42:47.088246 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:47.088249 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:47.088459 (+ 210us) server_negotiation.cc:407] Connection header received
0928 14:42:47.088927 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:47.088928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:47.088941 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:47.089024 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:47.089560 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.090419 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.091532 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.091824 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.091909 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.094486 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:47.094488 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:47.094488 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:47.094496 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:47.096207 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:47.096660 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:47.096662 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:47.096663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:47.096727 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:47.097003 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:47.097004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:47.097004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:47.097149 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:47.097211 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:47.097493 (+ 282us) server_negotiation.cc:299] Negotiation successful
0928 14:42:47.097596 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260928 14:42:47.097867 10560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.088150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37901 (local address 127.9.33.65:40645)
0928 14:42:47.088298 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:47.088363 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:47.088831 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:47.089075 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:47.089078 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:47.089299 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:47.089485 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:47.089489 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.090550 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.090551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:47.091420 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:47.091423 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.091920 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.091921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:47.092054 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.092604 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:47.092619 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:47.094383 (+ 1764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:47.096301 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:47.096334 (+ 33us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:47.096337 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:47.096566 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:47.096809 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:47.096810 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:47.096810 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:47.096919 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:47.097203 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:47.097209 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:47.097357 (+ 148us) client_negotiation.cc:769] Sending connection context
0928 14:42:47.097472 (+ 115us) client_negotiation.cc:240] Negotiation successful
0928 14:42:47.097628 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":86,"threads_started":1}
I20260928 14:42:47.098384 10667 heartbeater.cc:347] Connected to a master server at 127.9.33.126:37901
I20260928 14:42:47.098481 10667 heartbeater.cc:464] Registering TS with master...
I20260928 14:42:47.098706 10667 heartbeater.cc:511] Master 127.9.33.126:37901 requested a full tablet report, sending...
I20260928 14:42:47.099291 10463 ts_manager.cc:194] Registered new tserver with Master: 887d7faee69d4abea170536a8cdde4ce (127.9.33.65:37863)
I20260928 14:42:47.100234 10463 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.65@KRBTEST.COM'} at 127.9.33.65:40645
I20260928 14:42:47.105516 10514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.099563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:50083 (local address 127.9.33.126:37901)
0928 14:42:47.099599 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:47.099601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:47.099612 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:42:47.099648 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:47.099649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:47.099660 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:47.099712 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:47.100103 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.100774 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.101591 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.101815 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.101889 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.102963 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:47.102964 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:47.102964 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:47.102970 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:47.104322 (+ 1352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:47.104742 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:47.104743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:47.104744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:47.104793 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:47.105023 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:47.105024 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:47.105025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:47.105136 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:47.105189 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:47.105389 (+ 200us) server_negotiation.cc:299] Negotiation successful
0928 14:42:47.105454 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260928 14:42:47.105579 10674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.099393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37901 (local address 127.9.33.65:50083)
0928 14:42:47.099525 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:47.099537 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:47.099591 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:47.099743 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:47.099744 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:47.099946 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:47.100020 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:47.100022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.100878 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.100880 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:47.101484 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:47.101486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.101908 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.101910 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:47.102020 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.102435 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:47.102440 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:47.102872 (+ 432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:47.104404 (+ 1532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:47.104405 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:47.104406 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:47.104639 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:47.104855 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:47.104856 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:47.104856 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:47.104927 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:47.105205 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:47.105206 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:47.105258 (+ 52us) client_negotiation.cc:769] Sending connection context
0928 14:42:47.105351 (+ 93us) client_negotiation.cc:240] Negotiation successful
0928 14:42:47.105463 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":75,"threads_started":1}
WARNING: no policy specified for kudu/127.9.33.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab.
Entry for principal kudu/127.9.33.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab.
I20260928 14:42:47.127671 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.66:0
--local_ip_for_outbound_sockets=127.9.33.66
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:37901
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:45333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:47.215022 10676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:47.215238 10676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:47.215266 10676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:47.216447 10676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:47.216490 10676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:47.216511 10676 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 14:42:47.216529 10676 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.66
I20260928 14:42:47.217936 10676 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:45333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.33.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.33.126:37901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.66
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:47.218130 10676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:47.218310 10676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:47.223551 10684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:47.223692 10676 server_base.cc:1034] running on GCE node
W20260928 14:42:47.223555 10682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:47.223640 10681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:47.224036 10676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:47.224255 10676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:47.225421 10676 hybrid_clock.cc:648] HybridClock initialized: now 1790606567225418 us; error 35 us; skew 500 ppm
Sep 28 14:42:47 dist-test-slave-pv0w krb5kdc[10419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606567, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:47.227917 10676 init.cc:377] Logged in from keytab as kudu/127.9.33.66@KRBTEST.COM (short username kudu)
I20260928 14:42:47.228363 10676 webserver.cc:460] Webserver started at http://127.9.33.66:35505/ using document root <none> and password file <none>
I20260928 14:42:47.228608 10676 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:47.228678 10676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:47.228832 10676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:47.229929 10676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-1/data/instance:
uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9"
format_stamp: "Formatted at 2026-09-28 14:42:47 on dist-test-slave-pv0w"
server_key: "ef376c2f0b7eeaeb8a80d09d7e3851fb"
server_key_iv: "69c3cad1753ca5c967c8b58891924e1a"
server_key_version: "encryptionkey@0"
I20260928 14:42:47.230324 10676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance:
uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9"
format_stamp: "Formatted at 2026-09-28 14:42:47 on dist-test-slave-pv0w"
server_key: "ef376c2f0b7eeaeb8a80d09d7e3851fb"
server_key_iv: "69c3cad1753ca5c967c8b58891924e1a"
server_key_version: "encryptionkey@0"
I20260928 14:42:47.231909 10676 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:42:47.232810 10691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:47.233085 10676 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:42:47.233188 10676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-1/wal
uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9"
format_stamp: "Formatted at 2026-09-28 14:42:47 on dist-test-slave-pv0w"
server_key: "ef376c2f0b7eeaeb8a80d09d7e3851fb"
server_key_iv: "69c3cad1753ca5c967c8b58891924e1a"
server_key_version: "encryptionkey@0"
I20260928 14:42:47.233282 10676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:47.268718 10676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:47.271266 10676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:47.271430 10676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:47.272049 10676 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:42:47.272102 10676 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:47.272130 10676 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:42:47.272143 10676 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 14:42:47 dist-test-slave-pv0w krb5kdc[10419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606567, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:42:47.279935 10676 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.66:45309
I20260928 14:42:47.280020 10806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.66:45309 every 8 connection(s)
I20260928 14:42:47.280359 10676 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
I20260928 14:42:47.282874 10700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.272666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37901 (local address 127.9.33.66:54391)
0928 14:42:47.272851 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:47.272887 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:47.273455 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:47.273802 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:47.273806 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:47.274186 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:47.274505 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:47.274513 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.275612 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.275613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:47.276513 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:47.276516 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.276946 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.276947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:47.277071 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.277594 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:47.277602 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:47.279555 (+ 1953us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:47.281342 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:47.281347 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:47.281349 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:47.281564 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:47.281809 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:47.281810 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:47.281810 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:47.281908 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:47.282331 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:47.282337 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:47.282463 (+ 126us) client_negotiation.cc:769] Sending connection context
0928 14:42:47.282562 (+ 99us) client_negotiation.cc:240] Negotiation successful
0928 14:42:47.282697 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":105,"threads_started":1}
I20260928 14:42:47.282827 10514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.272665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:54391 (local address 127.9.33.126:37901)
0928 14:42:47.272736 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:47.272738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:47.272997 (+ 259us) server_negotiation.cc:407] Connection header received
0928 14:42:47.273590 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:47.273591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:47.273608 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:47.273687 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:47.274615 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.275500 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.276612 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.276833 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.276925 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.279683 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:47.279685 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:47.279685 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:47.279693 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:47.281267 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:47.281660 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:47.281661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:47.281662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:47.281737 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:47.281971 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:47.281972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:47.281972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:47.282254 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:47.282322 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:47.282661 (+ 339us) server_negotiation.cc:299] Negotiation successful
0928 14:42:47.282749 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 14:42:47.283455 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 10676
I20260928 14:42:47.283533 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance
I20260928 14:42:47.283666 9349 external_mini_cluster.cc:1428] Setting key c51d46052154c0c1a0aafab754127bd1
I20260928 14:42:47.284101 10807 heartbeater.cc:347] Connected to a master server at 127.9.33.126:37901
I20260928 14:42:47.284215 10807 heartbeater.cc:464] Registering TS with master...
I20260928 14:42:47.284464 10807 heartbeater.cc:511] Master 127.9.33.126:37901 requested a full tablet report, sending...
I20260928 14:42:47.284821 10463 ts_manager.cc:194] Registered new tserver with Master: e7c0ff7ea89943d8a2cb6de76fc289d9 (127.9.33.66:45309)
I20260928 14:42:47.285274 10463 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.66@KRBTEST.COM'} at 127.9.33.66:54391
I20260928 14:42:47.291282 10514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.285437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:49307 (local address 127.9.33.126:37901)
0928 14:42:47.285498 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:47.285500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:47.285508 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:42:47.285527 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:47.285528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:47.285539 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:47.285600 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:47.286059 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.286578 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.287282 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.287492 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.287568 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.288615 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:47.288617 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:47.288618 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:47.288625 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:47.290047 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:47.290423 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:47.290424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:47.290425 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:47.290484 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:47.290686 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:47.290687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:47.290688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:47.290853 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:47.290928 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:47.291096 (+ 168us) server_negotiation.cc:299] Negotiation successful
0928 14:42:47.291192 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260928 14:42:47.291290 10813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.285256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37901 (local address 127.9.33.66:49307)
0928 14:42:47.285386 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:47.285398 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:47.285460 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:47.285620 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:47.285621 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:47.285869 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:47.285955 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:47.285957 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.286669 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.286669 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:47.287193 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:47.287194 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.287566 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.287567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:47.287643 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.288198 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:47.288203 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:47.288505 (+ 302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:47.290152 (+ 1647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:47.290153 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:47.290154 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:47.290335 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:47.290554 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:47.290555 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:47.290555 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:47.290615 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:47.290930 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:47.290932 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:47.290981 (+ 49us) client_negotiation.cc:769] Sending connection context
0928 14:42:47.291068 (+ 87us) client_negotiation.cc:240] Negotiation successful
0928 14:42:47.291181 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":72,"threads_started":1}
WARNING: no policy specified for kudu/127.9.33.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab.
Entry for principal kudu/127.9.33.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab.
I20260928 14:42:47.311080 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.67:0
--local_ip_for_outbound_sockets=127.9.33.67
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:37901
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:45333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:47.402516 10816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:47.402765 10816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:47.402796 10816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:47.404175 10816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:47.404227 10816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:47.404256 10816 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 14:42:47.404277 10816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.67
I20260928 14:42:47.406157 10816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:45333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.33.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.33.126:37901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.67
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:47.406387 10816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:47.406597 10816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:47.412853 10822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:47.412853 10824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:47.412868 10821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:47.413187 10816 server_base.cc:1034] running on GCE node
I20260928 14:42:47.413437 10816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:47.413717 10816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:47.414956 10816 hybrid_clock.cc:648] HybridClock initialized: now 1790606567414935 us; error 29 us; skew 500 ppm
Sep 28 14:42:47 dist-test-slave-pv0w krb5kdc[10419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606567, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:47.417713 10816 init.cc:377] Logged in from keytab as kudu/127.9.33.67@KRBTEST.COM (short username kudu)
I20260928 14:42:47.418172 10816 webserver.cc:460] Webserver started at http://127.9.33.67:38371/ using document root <none> and password file <none>
I20260928 14:42:47.418440 10816 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:47.418514 10816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:47.418663 10816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:47.419845 10816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-2/data/instance:
uuid: "a5e464d260e84647a928ec1b15a5bc71"
format_stamp: "Formatted at 2026-09-28 14:42:47 on dist-test-slave-pv0w"
server_key: "40343feb87d0c9b1dd75c5f02b0d2809"
server_key_iv: "d692be3fb7f7d573bd23b4b2314028dc"
server_key_version: "encryptionkey@0"
I20260928 14:42:47.420222 10816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance:
uuid: "a5e464d260e84647a928ec1b15a5bc71"
format_stamp: "Formatted at 2026-09-28 14:42:47 on dist-test-slave-pv0w"
server_key: "40343feb87d0c9b1dd75c5f02b0d2809"
server_key_iv: "d692be3fb7f7d573bd23b4b2314028dc"
server_key_version: "encryptionkey@0"
I20260928 14:42:47.421857 10816 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260928 14:42:47.422762 10831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:47.422997 10816 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:42:47.423101 10816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-2/wal
uuid: "a5e464d260e84647a928ec1b15a5bc71"
format_stamp: "Formatted at 2026-09-28 14:42:47 on dist-test-slave-pv0w"
server_key: "40343feb87d0c9b1dd75c5f02b0d2809"
server_key_iv: "d692be3fb7f7d573bd23b4b2314028dc"
server_key_version: "encryptionkey@0"
I20260928 14:42:47.423192 10816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:47.452484 10816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:47.454919 10816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:47.455083 10816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:47.455616 10816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:42:47.455675 10816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:47.455796 10816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:42:47.455832 10816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:47.462783 10816 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.67:32995
I20260928 14:42:47.462917 10946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.67:32995 every 8 connection(s)
Sep 28 14:42:47 dist-test-slave-pv0w krb5kdc[10419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606567, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:42:47.463240 10816 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
I20260928 14:42:47.465968 10514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.456212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:37985 (local address 127.9.33.126:37901)
0928 14:42:47.456282 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:47.456284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:47.456507 (+ 223us) server_negotiation.cc:407] Connection header received
0928 14:42:47.457078 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:47.457080 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:47.457093 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:47.457176 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:47.458163 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.459025 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.460636 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.460950 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.461036 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.463552 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:47.463553 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:47.463554 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:47.463561 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:47.464826 (+ 1265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:47.465198 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:47.465199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:47.465199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:47.465249 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:47.465469 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:47.465470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:47.465470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:47.465568 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:47.465617 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:47.465840 (+ 223us) server_negotiation.cc:299] Negotiation successful
0928 14:42:47.465905 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 14:42:47.466131 10840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.456193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37901 (local address 127.9.33.67:37985)
0928 14:42:47.456374 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:47.456415 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:47.456980 (+ 565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:47.457259 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:47.457263 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:47.457740 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:47.458053 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:47.458061 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.459128 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.459129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:47.460525 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:47.460529 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.461063 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.461064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:47.461211 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.461820 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:47.461830 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:47.463442 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:47.464894 (+ 1452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:47.464900 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:47.464901 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:47.465119 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:47.465305 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:47.465305 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:47.465306 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:47.465415 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:47.465620 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:47.465624 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:47.465734 (+ 110us) client_negotiation.cc:769] Sending connection context
0928 14:42:47.465812 (+ 78us) client_negotiation.cc:240] Negotiation successful
0928 14:42:47.465932 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":110,"threads_started":1}
I20260928 14:42:47.466606 10947 heartbeater.cc:347] Connected to a master server at 127.9.33.126:37901
I20260928 14:42:47.466737 10947 heartbeater.cc:464] Registering TS with master...
I20260928 14:42:47.466847 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 10816
I20260928 14:42:47.466926 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance
I20260928 14:42:47.466940 10947 heartbeater.cc:511] Master 127.9.33.126:37901 requested a full tablet report, sending...
I20260928 14:42:47.467101 9349 external_mini_cluster.cc:1428] Setting key 6a1e15c1adfae39bf75fefda01270223
I20260928 14:42:47.467295 10464 ts_manager.cc:194] Registered new tserver with Master: a5e464d260e84647a928ec1b15a5bc71 (127.9.33.67:32995)
I20260928 14:42:47.467831 10464 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.67@KRBTEST.COM'} at 127.9.33.67:37985
I20260928 14:42:47.467938 9349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:42:47.475656 10514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.467736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:60229 (local address 127.9.33.126:37901)
0928 14:42:47.467770 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:47.467771 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:47.469571 (+ 1800us) server_negotiation.cc:407] Connection header received
0928 14:42:47.469617 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:47.469618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:47.469631 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:47.469702 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:47.469968 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.470692 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.471857 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.472038 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.472124 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.473091 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:47.473092 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:47.473093 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:47.473100 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:47.474502 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:47.474875 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:47.474876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:47.474877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:47.474933 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:47.475138 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:47.475139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:47.475140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:47.475253 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:47.475328 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:47.475487 (+ 159us) server_negotiation.cc:299] Negotiation successful
0928 14:42:47.475562 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260928 14:42:47.475668 10953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.467688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37901 (local address 127.9.33.67:60229)
0928 14:42:47.469513 (+ 1825us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:47.469524 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:47.469576 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:47.469705 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:47.469705 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:47.469851 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:47.469921 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:47.469922 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.470795 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.470796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:47.471774 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:47.471776 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.472122 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.472123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:47.472202 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.472666 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:47.472671 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:47.472986 (+ 315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:47.474604 (+ 1618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:47.474606 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:47.474606 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:47.474791 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:47.475007 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:47.475008 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:47.475008 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:47.475069 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:47.475319 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:47.475320 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:47.475376 (+ 56us) client_negotiation.cc:769] Sending connection context
0928 14:42:47.475459 (+ 83us) client_negotiation.cc:240] Negotiation successful
0928 14:42:47.475564 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":88,"threads_started":1}
Sep 28 14:42:47 dist-test-slave-pv0w krb5kdc[10419](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606567, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 28 14:42:47 dist-test-slave-pv0w krb5kdc[10419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606567, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:42:47.491236 10514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.484361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49212 (local address 127.9.33.126:37901)
0928 14:42:47.484426 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:47.484428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:47.484438 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:42:47.484465 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:47.484465 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:47.484473 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:47.484520 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:47.485113 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.485597 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.486375 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.486549 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.486598 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.488905 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:47.488907 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:47.488907 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:47.488913 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:47.490256 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:47.490557 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:47.490557 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:47.490558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:47.490607 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:47.490771 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:47.490771 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:47.490772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:47.490883 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:47.490956 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:47.491112 (+ 156us) server_negotiation.cc:299] Negotiation successful
0928 14:42:47.491171 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260928 14:42:47.493041 10464 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49212:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:42:47.494735 10464 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:42:47.501637 10965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.497214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49936 (local address 127.9.33.65:37863)
0928 14:42:47.497463 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:47.497469 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:47.497481 (+ 12us) server_negotiation.cc:407] Connection header received
0928 14:42:47.497523 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:47.497525 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:47.497543 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:47.497908 (+ 365us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:42:47.497976 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.499155 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.500538 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.501220 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.501286 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.501334 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:47.501490 (+ 156us) server_negotiation.cc:299] Negotiation successful
0928 14:42:47.501531 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":127,"threads_started":1}
I20260928 14:42:47.501641 10964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.497063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:37863 (local address 127.0.0.1:49936)
0928 14:42:47.497175 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:47.497219 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:47.497278 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:47.497677 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:47.497681 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:47.497739 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:42:47.497820 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:47.497822 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.499246 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.499247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:47.500452 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:47.500455 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.501308 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.501309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:47.501360 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.501382 (+ 22us) client_negotiation.cc:769] Sending connection context
0928 14:42:47.501465 (+ 83us) client_negotiation.cc:240] Negotiation successful
0928 14:42:47.501535 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":37,"thread_start_us":68,"threads_started":1}
I20260928 14:42:47.501942 10969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.497659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54704 (local address 127.9.33.67:32995)
0928 14:42:47.497868 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:47.497873 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:47.497881 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:42:47.497919 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:47.497921 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:47.497937 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:47.498011 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:42:47.498167 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.499249 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.500390 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.501141 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.501234 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.501302 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:47.501810 (+ 508us) server_negotiation.cc:299] Negotiation successful
0928 14:42:47.501851 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":71,"threads_started":1}
I20260928 14:42:47.501641 10968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.497460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:32995 (local address 127.0.0.1:54704)
0928 14:42:47.497551 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:47.497561 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:47.497614 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:47.498069 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:47.498070 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:47.498071 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:42:47.498104 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:47.498105 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.499348 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.499349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:47.500304 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:47.500307 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.501243 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.501244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:47.501360 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.501382 (+ 22us) client_negotiation.cc:769] Sending connection context
0928 14:42:47.501465 (+ 83us) client_negotiation.cc:240] Negotiation successful
0928 14:42:47.501535 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":2,"thread_start_us":63,"threads_started":1}
I20260928 14:42:47.502483 10601 tablet_service.cc:1467] Processing CreateTablet for tablet 62b52c5345c1435492c5ce32eaaa37f3 (DEFAULT_TABLE table=test-table [id=2bfdb004df1a4eefa56d7faf74c7125a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:42:47.502707 10966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.497286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:45309 (local address 127.0.0.1:60576)
0928 14:42:47.497400 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:47.497409 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:47.497446 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:47.497860 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:47.497861 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:47.497864 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:42:47.497924 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:47.497925 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.498991 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.498992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:47.500453 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:47.500455 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.502450 (+ 1995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.502450 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:47.502532 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.502536 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:42:47.502596 (+ 60us) client_negotiation.cc:240] Negotiation successful
0928 14:42:47.502635 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":32,"thread_start_us":83,"threads_started":1}
I20260928 14:42:47.502839 10967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.497402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60576 (local address 127.9.33.66:45309)
0928 14:42:47.497589 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:47.497593 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:47.497601 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:42:47.497638 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:47.497640 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:47.497655 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:47.497746 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:42:47.498013 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.498910 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.501369 (+ 2459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.502169 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.502253 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.502318 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:47.502617 (+ 299us) server_negotiation.cc:299] Negotiation successful
0928 14:42:47.502648 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":87,"threads_started":1}
I20260928 14:42:47.502920 10601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62b52c5345c1435492c5ce32eaaa37f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:47.504489 10881 tablet_service.cc:1467] Processing CreateTablet for tablet 62b52c5345c1435492c5ce32eaaa37f3 (DEFAULT_TABLE table=test-table [id=2bfdb004df1a4eefa56d7faf74c7125a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:42:47.504846 10881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62b52c5345c1435492c5ce32eaaa37f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:47.506642 10970 tablet_bootstrap.cc:492] T 62b52c5345c1435492c5ce32eaaa37f3 P 887d7faee69d4abea170536a8cdde4ce: Bootstrap starting.
I20260928 14:42:47.507229 10970 tablet_bootstrap.cc:654] T 62b52c5345c1435492c5ce32eaaa37f3 P 887d7faee69d4abea170536a8cdde4ce: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:47.507541 10970 log.cc:824] T 62b52c5345c1435492c5ce32eaaa37f3 P 887d7faee69d4abea170536a8cdde4ce: Log is configured to *not* fsync() on all Append() calls
I20260928 14:42:47.508401 10970 tablet_bootstrap.cc:492] T 62b52c5345c1435492c5ce32eaaa37f3 P 887d7faee69d4abea170536a8cdde4ce: No bootstrap required, opened a new log
I20260928 14:42:47.508473 10970 ts_tablet_manager.cc:1397] T 62b52c5345c1435492c5ce32eaaa37f3 P 887d7faee69d4abea170536a8cdde4ce: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260928 14:42:47.508509 10741 tablet_service.cc:1467] Processing CreateTablet for tablet 62b52c5345c1435492c5ce32eaaa37f3 (DEFAULT_TABLE table=test-table [id=2bfdb004df1a4eefa56d7faf74c7125a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:42:47.508845 10741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62b52c5345c1435492c5ce32eaaa37f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:47.510221 10970 raft_consensus.cc:348] T 62b52c5345c1435492c5ce32eaaa37f3 P 887d7faee69d4abea170536a8cdde4ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } }
I20260928 14:42:47.510377 10970 raft_consensus.cc:374] T 62b52c5345c1435492c5ce32eaaa37f3 P 887d7faee69d4abea170536a8cdde4ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:47.510462 10970 raft_consensus.cc:729] T 62b52c5345c1435492c5ce32eaaa37f3 P 887d7faee69d4abea170536a8cdde4ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 887d7faee69d4abea170536a8cdde4ce, State: Initialized, Role: FOLLOWER
I20260928 14:42:47.510629 10970 consensus_queue.cc:260] T 62b52c5345c1435492c5ce32eaaa37f3 P 887d7faee69d4abea170536a8cdde4ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } }
I20260928 14:42:47.510954 10970 ts_tablet_manager.cc:1428] T 62b52c5345c1435492c5ce32eaaa37f3 P 887d7faee69d4abea170536a8cdde4ce: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260928 14:42:47.510994 10667 heartbeater.cc:503] Master 127.9.33.126:37901 was elected leader, sending a full tablet report...
I20260928 14:42:47.511148 10972 tablet_bootstrap.cc:492] T 62b52c5345c1435492c5ce32eaaa37f3 P a5e464d260e84647a928ec1b15a5bc71: Bootstrap starting.
I20260928 14:42:47.512037 10972 tablet_bootstrap.cc:654] T 62b52c5345c1435492c5ce32eaaa37f3 P a5e464d260e84647a928ec1b15a5bc71: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:47.512169 10974 tablet_bootstrap.cc:492] T 62b52c5345c1435492c5ce32eaaa37f3 P e7c0ff7ea89943d8a2cb6de76fc289d9: Bootstrap starting.
I20260928 14:42:47.512390 10972 log.cc:824] T 62b52c5345c1435492c5ce32eaaa37f3 P a5e464d260e84647a928ec1b15a5bc71: Log is configured to *not* fsync() on all Append() calls
I20260928 14:42:47.512876 10974 tablet_bootstrap.cc:654] T 62b52c5345c1435492c5ce32eaaa37f3 P e7c0ff7ea89943d8a2cb6de76fc289d9: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:47.513170 10974 log.cc:824] T 62b52c5345c1435492c5ce32eaaa37f3 P e7c0ff7ea89943d8a2cb6de76fc289d9: Log is configured to *not* fsync() on all Append() calls
I20260928 14:42:47.513206 10972 tablet_bootstrap.cc:492] T 62b52c5345c1435492c5ce32eaaa37f3 P a5e464d260e84647a928ec1b15a5bc71: No bootstrap required, opened a new log
I20260928 14:42:47.513257 10972 ts_tablet_manager.cc:1397] T 62b52c5345c1435492c5ce32eaaa37f3 P a5e464d260e84647a928ec1b15a5bc71: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:42:47.513827 10974 tablet_bootstrap.cc:492] T 62b52c5345c1435492c5ce32eaaa37f3 P e7c0ff7ea89943d8a2cb6de76fc289d9: No bootstrap required, opened a new log
I20260928 14:42:47.513908 10974 ts_tablet_manager.cc:1397] T 62b52c5345c1435492c5ce32eaaa37f3 P e7c0ff7ea89943d8a2cb6de76fc289d9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:42:47.514781 10972 raft_consensus.cc:348] T 62b52c5345c1435492c5ce32eaaa37f3 P a5e464d260e84647a928ec1b15a5bc71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } }
I20260928 14:42:47.514918 10972 raft_consensus.cc:374] T 62b52c5345c1435492c5ce32eaaa37f3 P a5e464d260e84647a928ec1b15a5bc71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:47.514940 10972 raft_consensus.cc:729] T 62b52c5345c1435492c5ce32eaaa37f3 P a5e464d260e84647a928ec1b15a5bc71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5e464d260e84647a928ec1b15a5bc71, State: Initialized, Role: FOLLOWER
I20260928 14:42:47.515069 10972 consensus_queue.cc:260] T 62b52c5345c1435492c5ce32eaaa37f3 P a5e464d260e84647a928ec1b15a5bc71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } }
I20260928 14:42:47.515311 10972 ts_tablet_manager.cc:1428] T 62b52c5345c1435492c5ce32eaaa37f3 P a5e464d260e84647a928ec1b15a5bc71: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:42:47.515393 10947 heartbeater.cc:503] Master 127.9.33.126:37901 was elected leader, sending a full tablet report...
I20260928 14:42:47.515582 10974 raft_consensus.cc:348] T 62b52c5345c1435492c5ce32eaaa37f3 P e7c0ff7ea89943d8a2cb6de76fc289d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } }
I20260928 14:42:47.515766 10974 raft_consensus.cc:374] T 62b52c5345c1435492c5ce32eaaa37f3 P e7c0ff7ea89943d8a2cb6de76fc289d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:47.515818 10974 raft_consensus.cc:729] T 62b52c5345c1435492c5ce32eaaa37f3 P e7c0ff7ea89943d8a2cb6de76fc289d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7c0ff7ea89943d8a2cb6de76fc289d9, State: Initialized, Role: FOLLOWER
I20260928 14:42:47.515923 10974 consensus_queue.cc:260] T 62b52c5345c1435492c5ce32eaaa37f3 P e7c0ff7ea89943d8a2cb6de76fc289d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } }
I20260928 14:42:47.516197 10974 ts_tablet_manager.cc:1428] T 62b52c5345c1435492c5ce32eaaa37f3 P e7c0ff7ea89943d8a2cb6de76fc289d9: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260928 14:42:47.516304 10807 heartbeater.cc:503] Master 127.9.33.126:37901 was elected leader, sending a full tablet report...
W20260928 14:42:47.530974 10808 tablet.cc:2369] T 62b52c5345c1435492c5ce32eaaa37f3 P e7c0ff7ea89943d8a2cb6de76fc289d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:42:47.541466 10978 raft_consensus.cc:482] T 62b52c5345c1435492c5ce32eaaa37f3 P e7c0ff7ea89943d8a2cb6de76fc289d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:42:47.541600 10978 raft_consensus.cc:504] T 62b52c5345c1435492c5ce32eaaa37f3 P e7c0ff7ea89943d8a2cb6de76fc289d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } }
I20260928 14:42:47.541944 10978 leader_election.cc:288] T 62b52c5345c1435492c5ce32eaaa37f3 P e7c0ff7ea89943d8a2cb6de76fc289d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 887d7faee69d4abea170536a8cdde4ce (127.9.33.65:37863), a5e464d260e84647a928ec1b15a5bc71 (127.9.33.67:32995)
I20260928 14:42:47.546310 10965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.542302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:59037 (local address 127.9.33.65:37863)
0928 14:42:47.542371 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:47.542373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:47.542382 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:42:47.542411 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:47.542412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:47.542423 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:47.542496 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:42:47.542691 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.543562 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.545287 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.545888 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.545967 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.545976 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:47.546172 (+ 196us) server_negotiation.cc:299] Negotiation successful
0928 14:42:47.546229 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260928 14:42:47.546307 10969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.542440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:41313 (local address 127.9.33.67:32995)
0928 14:42:47.542483 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:47.542485 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:47.542533 (+ 48us) server_negotiation.cc:407] Connection header received
0928 14:42:47.542631 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:47.542632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:47.542642 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:47.542719 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:42:47.542904 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.543615 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.545287 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.545845 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.545929 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.545943 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:47.546172 (+ 229us) server_negotiation.cc:299] Negotiation successful
0928 14:42:47.546233 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 14:42:47.546281 10700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.542163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:37863 (local address 127.9.33.66:59037)
0928 14:42:47.542229 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:47.542242 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:47.542289 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:47.542530 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:47.542531 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:47.542533 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:42:47.542610 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:47.542612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.543702 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.543703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:47.545164 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:47.545167 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.545977 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.545978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:47.546054 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.546056 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 14:42:47.546117 (+ 61us) client_negotiation.cc:240] Negotiation successful
0928 14:42:47.546178 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":137}
I20260928 14:42:47.546285 10979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.542279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:32995 (local address 127.9.33.66:41313)
0928 14:42:47.542479 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:47.542489 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:47.542557 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:47.542774 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:47.542775 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:47.542777 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:42:47.542827 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:47.542828 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.543703 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.543703 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:47.545164 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:47.545167 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.545954 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.545955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:47.546051 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.546055 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:42:47.546117 (+ 62us) client_negotiation.cc:240] Negotiation successful
0928 14:42:47.546178 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":81,"thread_start_us":116,"threads_started":1}
I20260928 14:42:47.546739 10619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62b52c5345c1435492c5ce32eaaa37f3" candidate_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "887d7faee69d4abea170536a8cdde4ce" is_pre_election: true
I20260928 14:42:47.546800 10901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62b52c5345c1435492c5ce32eaaa37f3" candidate_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5e464d260e84647a928ec1b15a5bc71" is_pre_election: true
I20260928 14:42:47.546934 10619 raft_consensus.cc:2454] T 62b52c5345c1435492c5ce32eaaa37f3 P 887d7faee69d4abea170536a8cdde4ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7c0ff7ea89943d8a2cb6de76fc289d9 in term 0.
I20260928 14:42:47.547003 10901 raft_consensus.cc:2454] T 62b52c5345c1435492c5ce32eaaa37f3 P a5e464d260e84647a928ec1b15a5bc71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7c0ff7ea89943d8a2cb6de76fc289d9 in term 0.
I20260928 14:42:47.547201 10695 leader_election.cc:302] T 62b52c5345c1435492c5ce32eaaa37f3 P e7c0ff7ea89943d8a2cb6de76fc289d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 887d7faee69d4abea170536a8cdde4ce, e7c0ff7ea89943d8a2cb6de76fc289d9; no voters:
I20260928 14:42:47.547370 10978 raft_consensus.cc:2781] T 62b52c5345c1435492c5ce32eaaa37f3 P e7c0ff7ea89943d8a2cb6de76fc289d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:42:47.547459 10978 raft_consensus.cc:482] T 62b52c5345c1435492c5ce32eaaa37f3 P e7c0ff7ea89943d8a2cb6de76fc289d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:42:47.547479 10978 raft_consensus.cc:3037] T 62b52c5345c1435492c5ce32eaaa37f3 P e7c0ff7ea89943d8a2cb6de76fc289d9 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:47.548799 10978 raft_consensus.cc:504] T 62b52c5345c1435492c5ce32eaaa37f3 P e7c0ff7ea89943d8a2cb6de76fc289d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } }
I20260928 14:42:47.548941 10978 leader_election.cc:288] T 62b52c5345c1435492c5ce32eaaa37f3 P e7c0ff7ea89943d8a2cb6de76fc289d9 [CANDIDATE]: Term 1 election: Requested vote from peers 887d7faee69d4abea170536a8cdde4ce (127.9.33.65:37863), a5e464d260e84647a928ec1b15a5bc71 (127.9.33.67:32995)
I20260928 14:42:47.549194 10619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62b52c5345c1435492c5ce32eaaa37f3" candidate_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "887d7faee69d4abea170536a8cdde4ce"
I20260928 14:42:47.549245 10901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62b52c5345c1435492c5ce32eaaa37f3" candidate_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5e464d260e84647a928ec1b15a5bc71"
I20260928 14:42:47.549335 10901 raft_consensus.cc:3037] T 62b52c5345c1435492c5ce32eaaa37f3 P a5e464d260e84647a928ec1b15a5bc71 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:47.549340 10619 raft_consensus.cc:3037] T 62b52c5345c1435492c5ce32eaaa37f3 P 887d7faee69d4abea170536a8cdde4ce [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:47.550263 10901 raft_consensus.cc:2454] T 62b52c5345c1435492c5ce32eaaa37f3 P a5e464d260e84647a928ec1b15a5bc71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7c0ff7ea89943d8a2cb6de76fc289d9 in term 1.
I20260928 14:42:47.550275 10619 raft_consensus.cc:2454] T 62b52c5345c1435492c5ce32eaaa37f3 P 887d7faee69d4abea170536a8cdde4ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7c0ff7ea89943d8a2cb6de76fc289d9 in term 1.
I20260928 14:42:47.550462 10695 leader_election.cc:302] T 62b52c5345c1435492c5ce32eaaa37f3 P e7c0ff7ea89943d8a2cb6de76fc289d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 887d7faee69d4abea170536a8cdde4ce, e7c0ff7ea89943d8a2cb6de76fc289d9; no voters:
I20260928 14:42:47.550585 10978 raft_consensus.cc:2781] T 62b52c5345c1435492c5ce32eaaa37f3 P e7c0ff7ea89943d8a2cb6de76fc289d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:42:47.550766 10978 raft_consensus.cc:686] T 62b52c5345c1435492c5ce32eaaa37f3 P e7c0ff7ea89943d8a2cb6de76fc289d9 [term 1 LEADER]: Becoming Leader. State: Replica: e7c0ff7ea89943d8a2cb6de76fc289d9, State: Running, Role: LEADER
I20260928 14:42:47.550880 10978 consensus_queue.cc:237] T 62b52c5345c1435492c5ce32eaaa37f3 P e7c0ff7ea89943d8a2cb6de76fc289d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } }
I20260928 14:42:47.551822 10464 catalog_manager.cc:5495] T 62b52c5345c1435492c5ce32eaaa37f3 P e7c0ff7ea89943d8a2cb6de76fc289d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e7c0ff7ea89943d8a2cb6de76fc289d9 (127.9.33.66). New cstate: current_term: 1 leader_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } health_report { overall_health: UNKNOWN } } }
I20260928 14:42:47.560564 10967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.557569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60586 (local address 127.9.33.66:45309)
0928 14:42:47.557633 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:47.557635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:47.557645 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:42:47.557827 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:47.557828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:47.557855 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:47.557934 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:42:47.558097 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.558619 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.559521 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.559807 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.559899 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.560158 (+ 259us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:42:47.560270 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:42:47.560373 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:47.560421 (+ 48us) server_negotiation.cc:299] Negotiation successful
0928 14:42:47.560446 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260928 14:42:47.562448 10901 raft_consensus.cc:1261] T 62b52c5345c1435492c5ce32eaaa37f3 P a5e464d260e84647a928ec1b15a5bc71 [term 1 FOLLOWER]: Refusing update from remote peer e7c0ff7ea89943d8a2cb6de76fc289d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:42:47.562533 10619 raft_consensus.cc:1261] T 62b52c5345c1435492c5ce32eaaa37f3 P 887d7faee69d4abea170536a8cdde4ce [term 1 FOLLOWER]: Refusing update from remote peer e7c0ff7ea89943d8a2cb6de76fc289d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:42:47.562839 10978 consensus_queue.cc:1035] T 62b52c5345c1435492c5ce32eaaa37f3 P e7c0ff7ea89943d8a2cb6de76fc289d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:42:47.563005 10980 consensus_queue.cc:1035] T 62b52c5345c1435492c5ce32eaaa37f3 P e7c0ff7ea89943d8a2cb6de76fc289d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:42:47.566195 10985 mvcc.cc:205] Tried to move back new op lower bound from 7334324500733095936 to 7334324500689072128. Current Snapshot: MvccSnapshot[applied={T|T < 7334324500733095936}]
I20260928 14:42:47.570221 10463 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:42:47.570338 10463 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:42:47.571172 10463 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 990c5809ef95459b882bfeb2232d16b5: Sending DeleteTablet for 3 replicas of tablet 62b52c5345c1435492c5ce32eaaa37f3
I20260928 14:42:47.571524 10601 tablet_service.cc:1514] Processing DeleteTablet for tablet 62b52c5345c1435492c5ce32eaaa37f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:42:47 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:49936
I20260928 14:42:47.571571 10881 tablet_service.cc:1514] Processing DeleteTablet for tablet 62b52c5345c1435492c5ce32eaaa37f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:42:47 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:54704
I20260928 14:42:47.571614 10741 tablet_service.cc:1514] Processing DeleteTablet for tablet 62b52c5345c1435492c5ce32eaaa37f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:42:47 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:60576
I20260928 14:42:47.572022 10993 tablet_replica.cc:330] T 62b52c5345c1435492c5ce32eaaa37f3 P 887d7faee69d4abea170536a8cdde4ce: stopping tablet replica
I20260928 14:42:47.572142 10993 raft_consensus.cc:2229] T 62b52c5345c1435492c5ce32eaaa37f3 P 887d7faee69d4abea170536a8cdde4ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:42:47.572142 10463 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49212:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260928 14:42:47.572258 10993 raft_consensus.cc:2258] T 62b52c5345c1435492c5ce32eaaa37f3 P 887d7faee69d4abea170536a8cdde4ce [term 1 FOLLOWER]: Raft consensus is shut down!
W20260928 14:42:47.572274 10463 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:42:47.572420 10995 tablet_replica.cc:330] T 62b52c5345c1435492c5ce32eaaa37f3 P e7c0ff7ea89943d8a2cb6de76fc289d9: stopping tablet replica
I20260928 14:42:47.572518 10995 raft_consensus.cc:2229] T 62b52c5345c1435492c5ce32eaaa37f3 P e7c0ff7ea89943d8a2cb6de76fc289d9 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:42:47.572641 10995 raft_consensus.cc:2258] T 62b52c5345c1435492c5ce32eaaa37f3 P e7c0ff7ea89943d8a2cb6de76fc289d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:42:47.573578 10993 ts_tablet_manager.cc:1905] T 62b52c5345c1435492c5ce32eaaa37f3 P 887d7faee69d4abea170536a8cdde4ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:42:47.573716 10881 tablet_service.cc:1467] Processing CreateTablet for tablet 39dfe83205f84f6da2bfe080d3a908c6 (DEFAULT_TABLE table=test-table [id=315791256d584fa081a2d2d793896e1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:42:47.573791 10741 tablet_service.cc:1467] Processing CreateTablet for tablet 39dfe83205f84f6da2bfe080d3a908c6 (DEFAULT_TABLE table=test-table [id=315791256d584fa081a2d2d793896e1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:42:47.573846 10881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39dfe83205f84f6da2bfe080d3a908c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:47.573912 10741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39dfe83205f84f6da2bfe080d3a908c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:47.574028 10601 tablet_service.cc:1467] Processing CreateTablet for tablet 39dfe83205f84f6da2bfe080d3a908c6 (DEFAULT_TABLE table=test-table [id=315791256d584fa081a2d2d793896e1d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:42:47.574115 10601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39dfe83205f84f6da2bfe080d3a908c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:47.575488 10974 tablet_bootstrap.cc:492] T 39dfe83205f84f6da2bfe080d3a908c6 P e7c0ff7ea89943d8a2cb6de76fc289d9: Bootstrap starting.
I20260928 14:42:47.575636 10972 tablet_bootstrap.cc:492] T 39dfe83205f84f6da2bfe080d3a908c6 P a5e464d260e84647a928ec1b15a5bc71: Bootstrap starting.
I20260928 14:42:47.575814 10970 tablet_bootstrap.cc:492] T 39dfe83205f84f6da2bfe080d3a908c6 P 887d7faee69d4abea170536a8cdde4ce: Bootstrap starting.
I20260928 14:42:47.576113 10974 tablet_bootstrap.cc:654] T 39dfe83205f84f6da2bfe080d3a908c6 P e7c0ff7ea89943d8a2cb6de76fc289d9: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:47.576242 10972 tablet_bootstrap.cc:654] T 39dfe83205f84f6da2bfe080d3a908c6 P a5e464d260e84647a928ec1b15a5bc71: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:47.576241 10970 tablet_bootstrap.cc:654] T 39dfe83205f84f6da2bfe080d3a908c6 P 887d7faee69d4abea170536a8cdde4ce: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:47.576846 10974 tablet_bootstrap.cc:492] T 39dfe83205f84f6da2bfe080d3a908c6 P e7c0ff7ea89943d8a2cb6de76fc289d9: No bootstrap required, opened a new log
I20260928 14:42:47.576906 10974 ts_tablet_manager.cc:1397] T 39dfe83205f84f6da2bfe080d3a908c6 P e7c0ff7ea89943d8a2cb6de76fc289d9: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260928 14:42:47.577054 10974 raft_consensus.cc:348] T 39dfe83205f84f6da2bfe080d3a908c6 P e7c0ff7ea89943d8a2cb6de76fc289d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } }
I20260928 14:42:47.577239 10974 raft_consensus.cc:374] T 39dfe83205f84f6da2bfe080d3a908c6 P e7c0ff7ea89943d8a2cb6de76fc289d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:47.577266 10974 raft_consensus.cc:729] T 39dfe83205f84f6da2bfe080d3a908c6 P e7c0ff7ea89943d8a2cb6de76fc289d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7c0ff7ea89943d8a2cb6de76fc289d9, State: Initialized, Role: FOLLOWER
I20260928 14:42:47.577337 10974 consensus_queue.cc:260] T 39dfe83205f84f6da2bfe080d3a908c6 P e7c0ff7ea89943d8a2cb6de76fc289d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } }
I20260928 14:42:47.577440 10974 ts_tablet_manager.cc:1428] T 39dfe83205f84f6da2bfe080d3a908c6 P e7c0ff7ea89943d8a2cb6de76fc289d9: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260928 14:42:47.578217 10994 tablet_replica.cc:330] T 62b52c5345c1435492c5ce32eaaa37f3 P a5e464d260e84647a928ec1b15a5bc71: stopping tablet replica
I20260928 14:42:47.578341 10994 raft_consensus.cc:2229] T 62b52c5345c1435492c5ce32eaaa37f3 P a5e464d260e84647a928ec1b15a5bc71 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:42:47.578490 10994 raft_consensus.cc:2258] T 62b52c5345c1435492c5ce32eaaa37f3 P a5e464d260e84647a928ec1b15a5bc71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:42:47.578547 10995 ts_tablet_manager.cc:1905] T 62b52c5345c1435492c5ce32eaaa37f3 P e7c0ff7ea89943d8a2cb6de76fc289d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:42:47.578977 10994 ts_tablet_manager.cc:1905] T 62b52c5345c1435492c5ce32eaaa37f3 P a5e464d260e84647a928ec1b15a5bc71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:42:47.579164 10993 ts_tablet_manager.cc:1918] T 62b52c5345c1435492c5ce32eaaa37f3 P 887d7faee69d4abea170536a8cdde4ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:42:47.579284 10993 log.cc:1196] T 62b52c5345c1435492c5ce32eaaa37f3 P 887d7faee69d4abea170536a8cdde4ce: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-0/wal/wals/62b52c5345c1435492c5ce32eaaa37f3
I20260928 14:42:47.579645 10993 ts_tablet_manager.cc:1939] T 62b52c5345c1435492c5ce32eaaa37f3 P 887d7faee69d4abea170536a8cdde4ce: Deleting consensus metadata
I20260928 14:42:47.580237 10451 catalog_manager.cc:4841] TS 887d7faee69d4abea170536a8cdde4ce (127.9.33.65:37863): tablet 62b52c5345c1435492c5ce32eaaa37f3 (table test-table [id=2bfdb004df1a4eefa56d7faf74c7125a]) successfully deleted
I20260928 14:42:47.580644 10994 ts_tablet_manager.cc:1918] T 62b52c5345c1435492c5ce32eaaa37f3 P a5e464d260e84647a928ec1b15a5bc71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:42:47.580737 10994 log.cc:1196] T 62b52c5345c1435492c5ce32eaaa37f3 P a5e464d260e84647a928ec1b15a5bc71: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-2/wal/wals/62b52c5345c1435492c5ce32eaaa37f3
I20260928 14:42:47.580987 10994 ts_tablet_manager.cc:1939] T 62b52c5345c1435492c5ce32eaaa37f3 P a5e464d260e84647a928ec1b15a5bc71: Deleting consensus metadata
I20260928 14:42:47.581389 10451 catalog_manager.cc:4841] TS a5e464d260e84647a928ec1b15a5bc71 (127.9.33.67:32995): tablet 62b52c5345c1435492c5ce32eaaa37f3 (table test-table [id=2bfdb004df1a4eefa56d7faf74c7125a]) successfully deleted
I20260928 14:42:47.581849 10995 ts_tablet_manager.cc:1918] T 62b52c5345c1435492c5ce32eaaa37f3 P e7c0ff7ea89943d8a2cb6de76fc289d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:42:47.581930 10995 log.cc:1196] T 62b52c5345c1435492c5ce32eaaa37f3 P e7c0ff7ea89943d8a2cb6de76fc289d9: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-1/wal/wals/62b52c5345c1435492c5ce32eaaa37f3
I20260928 14:42:47.582193 10995 ts_tablet_manager.cc:1939] T 62b52c5345c1435492c5ce32eaaa37f3 P e7c0ff7ea89943d8a2cb6de76fc289d9: Deleting consensus metadata
I20260928 14:42:47.582634 10451 catalog_manager.cc:4841] TS e7c0ff7ea89943d8a2cb6de76fc289d9 (127.9.33.66:45309): tablet 62b52c5345c1435492c5ce32eaaa37f3 (table test-table [id=2bfdb004df1a4eefa56d7faf74c7125a]) successfully deleted
I20260928 14:42:47.584223 10970 tablet_bootstrap.cc:492] T 39dfe83205f84f6da2bfe080d3a908c6 P 887d7faee69d4abea170536a8cdde4ce: No bootstrap required, opened a new log
I20260928 14:42:47.584285 10970 ts_tablet_manager.cc:1397] T 39dfe83205f84f6da2bfe080d3a908c6 P 887d7faee69d4abea170536a8cdde4ce: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20260928 14:42:47.584446 10970 raft_consensus.cc:348] T 39dfe83205f84f6da2bfe080d3a908c6 P 887d7faee69d4abea170536a8cdde4ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } }
I20260928 14:42:47.584537 10970 raft_consensus.cc:374] T 39dfe83205f84f6da2bfe080d3a908c6 P 887d7faee69d4abea170536a8cdde4ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:47.584561 10970 raft_consensus.cc:729] T 39dfe83205f84f6da2bfe080d3a908c6 P 887d7faee69d4abea170536a8cdde4ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 887d7faee69d4abea170536a8cdde4ce, State: Initialized, Role: FOLLOWER
I20260928 14:42:47.584616 10970 consensus_queue.cc:260] T 39dfe83205f84f6da2bfe080d3a908c6 P 887d7faee69d4abea170536a8cdde4ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } }
I20260928 14:42:47.584721 10970 ts_tablet_manager.cc:1428] T 39dfe83205f84f6da2bfe080d3a908c6 P 887d7faee69d4abea170536a8cdde4ce: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:47.585111 10972 tablet_bootstrap.cc:492] T 39dfe83205f84f6da2bfe080d3a908c6 P a5e464d260e84647a928ec1b15a5bc71: No bootstrap required, opened a new log
I20260928 14:42:47.585165 10972 ts_tablet_manager.cc:1397] T 39dfe83205f84f6da2bfe080d3a908c6 P a5e464d260e84647a928ec1b15a5bc71: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.001s
I20260928 14:42:47.585319 10972 raft_consensus.cc:348] T 39dfe83205f84f6da2bfe080d3a908c6 P a5e464d260e84647a928ec1b15a5bc71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } }
I20260928 14:42:47.585400 10972 raft_consensus.cc:374] T 39dfe83205f84f6da2bfe080d3a908c6 P a5e464d260e84647a928ec1b15a5bc71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:47.585417 10972 raft_consensus.cc:729] T 39dfe83205f84f6da2bfe080d3a908c6 P a5e464d260e84647a928ec1b15a5bc71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5e464d260e84647a928ec1b15a5bc71, State: Initialized, Role: FOLLOWER
I20260928 14:42:47.585471 10972 consensus_queue.cc:260] T 39dfe83205f84f6da2bfe080d3a908c6 P a5e464d260e84647a928ec1b15a5bc71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } }
I20260928 14:42:47.585597 10972 ts_tablet_manager.cc:1428] T 39dfe83205f84f6da2bfe080d3a908c6 P a5e464d260e84647a928ec1b15a5bc71: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260928 14:42:47.596350 10668 tablet.cc:2369] T 39dfe83205f84f6da2bfe080d3a908c6 P 887d7faee69d4abea170536a8cdde4ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:42:47.713888 10948 tablet.cc:2369] T 39dfe83205f84f6da2bfe080d3a908c6 P a5e464d260e84647a928ec1b15a5bc71: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:42:47.715220 10973 raft_consensus.cc:482] T 39dfe83205f84f6da2bfe080d3a908c6 P 887d7faee69d4abea170536a8cdde4ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:42:47.715309 10973 raft_consensus.cc:504] T 39dfe83205f84f6da2bfe080d3a908c6 P 887d7faee69d4abea170536a8cdde4ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } }
I20260928 14:42:47.715627 10973 leader_election.cc:288] T 39dfe83205f84f6da2bfe080d3a908c6 P 887d7faee69d4abea170536a8cdde4ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7c0ff7ea89943d8a2cb6de76fc289d9 (127.9.33.66:45309), a5e464d260e84647a928ec1b15a5bc71 (127.9.33.67:32995)
I20260928 14:42:47.718722 10969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.716239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:32897 (local address 127.9.33.67:32995)
0928 14:42:47.716300 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:47.716304 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:47.716320 (+ 16us) server_negotiation.cc:407] Connection header received
0928 14:42:47.716345 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:47.716346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:47.716352 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:47.716396 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:42:47.716564 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.717050 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.718014 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.718379 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.718462 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.718472 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:47.718625 (+ 153us) server_negotiation.cc:299] Negotiation successful
0928 14:42:47.718657 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260928 14:42:47.718775 11000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.716146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:32995 (local address 127.9.33.65:32897)
0928 14:42:47.716224 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:47.716241 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:47.716304 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:47.716419 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:47.716420 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:47.716423 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:42:47.716482 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:47.716484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.717138 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.717139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:47.717915 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:47.717917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.718473 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.718474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:47.718542 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.718546 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:42:47.718607 (+ 61us) client_negotiation.cc:240] Negotiation successful
0928 14:42:47.718651 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"thread_start_us":40,"threads_started":1}
I20260928 14:42:47.718777 10967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.716002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:34821 (local address 127.9.33.66:45309)
0928 14:42:47.716093 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:47.716095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:47.716428 (+ 333us) server_negotiation.cc:407] Connection header received
0928 14:42:47.716450 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:47.716451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:47.716461 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:47.716525 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:42:47.716689 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.717150 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.718064 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.718426 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.718509 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.718519 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:47.718675 (+ 156us) server_negotiation.cc:299] Negotiation successful
0928 14:42:47.718711 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260928 14:42:47.718839 10999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.715914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:45309 (local address 127.9.33.65:34821)
0928 14:42:47.716097 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:47.716109 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:47.716215 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:47.716532 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:47.716533 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:47.716535 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:42:47.716597 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:47.716600 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.717238 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.717239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:47.717970 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:47.717972 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.718513 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.718514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:47.718584 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.718590 (+ 6us) client_negotiation.cc:769] Sending connection context
0928 14:42:47.718657 (+ 67us) client_negotiation.cc:240] Negotiation successful
0928 14:42:47.718702 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":95,"threads_started":1}
I20260928 14:42:47.719115 10901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39dfe83205f84f6da2bfe080d3a908c6" candidate_uuid: "887d7faee69d4abea170536a8cdde4ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5e464d260e84647a928ec1b15a5bc71" is_pre_election: true
I20260928 14:42:47.719110 10761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39dfe83205f84f6da2bfe080d3a908c6" candidate_uuid: "887d7faee69d4abea170536a8cdde4ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" is_pre_election: true
I20260928 14:42:47.719233 10901 raft_consensus.cc:2454] T 39dfe83205f84f6da2bfe080d3a908c6 P a5e464d260e84647a928ec1b15a5bc71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 887d7faee69d4abea170536a8cdde4ce in term 0.
I20260928 14:42:47.719241 10761 raft_consensus.cc:2454] T 39dfe83205f84f6da2bfe080d3a908c6 P e7c0ff7ea89943d8a2cb6de76fc289d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 887d7faee69d4abea170536a8cdde4ce in term 0.
I20260928 14:42:47.719470 10555 leader_election.cc:302] T 39dfe83205f84f6da2bfe080d3a908c6 P 887d7faee69d4abea170536a8cdde4ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 887d7faee69d4abea170536a8cdde4ce, e7c0ff7ea89943d8a2cb6de76fc289d9; no voters:
I20260928 14:42:47.719604 10973 raft_consensus.cc:2781] T 39dfe83205f84f6da2bfe080d3a908c6 P 887d7faee69d4abea170536a8cdde4ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:42:47.719673 10973 raft_consensus.cc:482] T 39dfe83205f84f6da2bfe080d3a908c6 P 887d7faee69d4abea170536a8cdde4ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:42:47.719703 10973 raft_consensus.cc:3037] T 39dfe83205f84f6da2bfe080d3a908c6 P 887d7faee69d4abea170536a8cdde4ce [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:47.720405 10973 raft_consensus.cc:504] T 39dfe83205f84f6da2bfe080d3a908c6 P 887d7faee69d4abea170536a8cdde4ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } }
I20260928 14:42:47.720548 10973 leader_election.cc:288] T 39dfe83205f84f6da2bfe080d3a908c6 P 887d7faee69d4abea170536a8cdde4ce [CANDIDATE]: Term 1 election: Requested vote from peers e7c0ff7ea89943d8a2cb6de76fc289d9 (127.9.33.66:45309), a5e464d260e84647a928ec1b15a5bc71 (127.9.33.67:32995)
I20260928 14:42:47.720728 10761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39dfe83205f84f6da2bfe080d3a908c6" candidate_uuid: "887d7faee69d4abea170536a8cdde4ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9"
I20260928 14:42:47.720778 10901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39dfe83205f84f6da2bfe080d3a908c6" candidate_uuid: "887d7faee69d4abea170536a8cdde4ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5e464d260e84647a928ec1b15a5bc71"
I20260928 14:42:47.720829 10761 raft_consensus.cc:3037] T 39dfe83205f84f6da2bfe080d3a908c6 P e7c0ff7ea89943d8a2cb6de76fc289d9 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:47.720860 10901 raft_consensus.cc:3037] T 39dfe83205f84f6da2bfe080d3a908c6 P a5e464d260e84647a928ec1b15a5bc71 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:47.721467 10901 raft_consensus.cc:2454] T 39dfe83205f84f6da2bfe080d3a908c6 P a5e464d260e84647a928ec1b15a5bc71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 887d7faee69d4abea170536a8cdde4ce in term 1.
I20260928 14:42:47.721499 10761 raft_consensus.cc:2454] T 39dfe83205f84f6da2bfe080d3a908c6 P e7c0ff7ea89943d8a2cb6de76fc289d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 887d7faee69d4abea170536a8cdde4ce in term 1.
I20260928 14:42:47.721658 10555 leader_election.cc:302] T 39dfe83205f84f6da2bfe080d3a908c6 P 887d7faee69d4abea170536a8cdde4ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 887d7faee69d4abea170536a8cdde4ce, a5e464d260e84647a928ec1b15a5bc71; no voters:
I20260928 14:42:47.721771 10973 raft_consensus.cc:2781] T 39dfe83205f84f6da2bfe080d3a908c6 P 887d7faee69d4abea170536a8cdde4ce [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:42:47.721869 10973 raft_consensus.cc:686] T 39dfe83205f84f6da2bfe080d3a908c6 P 887d7faee69d4abea170536a8cdde4ce [term 1 LEADER]: Becoming Leader. State: Replica: 887d7faee69d4abea170536a8cdde4ce, State: Running, Role: LEADER
I20260928 14:42:47.721949 10973 consensus_queue.cc:237] T 39dfe83205f84f6da2bfe080d3a908c6 P 887d7faee69d4abea170536a8cdde4ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } }
I20260928 14:42:47.722690 10463 catalog_manager.cc:5495] T 39dfe83205f84f6da2bfe080d3a908c6 P 887d7faee69d4abea170536a8cdde4ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 887d7faee69d4abea170536a8cdde4ce (127.9.33.65). New cstate: current_term: 1 leader_uuid: "887d7faee69d4abea170536a8cdde4ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } health_report { overall_health: UNKNOWN } } }
Sep 28 14:42:47 dist-test-slave-pv0w krb5kdc[10419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606566, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:42:47.788162 11008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.780634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37901 (local address 127.0.0.1:49228)
0928 14:42:47.780942 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:47.780952 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:47.781021 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:47.781146 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:47.781147 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:47.781483 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:47.781539 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:47.781540 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.782217 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.782218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:47.782852 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:47.782854 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.783168 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.783169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:47.783258 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.783984 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:47.783988 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:47.785616 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:47.787082 (+ 1466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:47.787090 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:47.787092 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:47.787300 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:47.787503 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:47.787503 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:47.787504 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:47.787561 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:47.787839 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:47.787845 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:47.787911 (+ 66us) client_negotiation.cc:769] Sending connection context
0928 14:42:47.787967 (+ 56us) client_negotiation.cc:240] Negotiation successful
0928 14:42:47.788064 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":229,"threads_started":1}
I20260928 14:42:47.788210 10514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.780739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49228 (local address 127.9.33.126:37901)
0928 14:42:47.780847 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:47.780849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:47.781021 (+ 172us) server_negotiation.cc:407] Connection header received
0928 14:42:47.781066 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:47.781067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:47.781076 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:47.781128 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:47.781631 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.782133 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.782926 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.783101 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.783151 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.785698 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:47.785700 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:47.785700 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:47.785706 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:47.786986 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:47.787385 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:47.787386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:47.787386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:47.787440 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:47.787617 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:47.787617 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:47.787618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:47.787781 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:47.787840 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:47.788004 (+ 164us) server_negotiation.cc:299] Negotiation successful
0928 14:42:47.788111 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":75}
I20260928 14:42:47.790472 10463 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:49228:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260928 14:42:47.790678 10463 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:42:47.792224 10741 tablet_service.cc:1467] Processing CreateTablet for tablet d1d9c30083b746248bc5dfdbb3899496 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=71b77393c1684bcf97df2ce2c618fc0e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 14:42:47.792240 10881 tablet_service.cc:1467] Processing CreateTablet for tablet d1d9c30083b746248bc5dfdbb3899496 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=71b77393c1684bcf97df2ce2c618fc0e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 14:42:47.792232 10601 tablet_service.cc:1467] Processing CreateTablet for tablet d1d9c30083b746248bc5dfdbb3899496 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=71b77393c1684bcf97df2ce2c618fc0e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 14:42:47.792407 10741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1d9c30083b746248bc5dfdbb3899496. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:47.792407 10601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1d9c30083b746248bc5dfdbb3899496. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:47.792414 10881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1d9c30083b746248bc5dfdbb3899496. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:47.793931 10974 tablet_bootstrap.cc:492] T d1d9c30083b746248bc5dfdbb3899496 P e7c0ff7ea89943d8a2cb6de76fc289d9: Bootstrap starting.
I20260928 14:42:47.794154 10972 tablet_bootstrap.cc:492] T d1d9c30083b746248bc5dfdbb3899496 P a5e464d260e84647a928ec1b15a5bc71: Bootstrap starting.
I20260928 14:42:47.794154 10970 tablet_bootstrap.cc:492] T d1d9c30083b746248bc5dfdbb3899496 P 887d7faee69d4abea170536a8cdde4ce: Bootstrap starting.
I20260928 14:42:47.794345 10974 tablet_bootstrap.cc:654] T d1d9c30083b746248bc5dfdbb3899496 P e7c0ff7ea89943d8a2cb6de76fc289d9: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:47.794847 10970 tablet_bootstrap.cc:654] T d1d9c30083b746248bc5dfdbb3899496 P 887d7faee69d4abea170536a8cdde4ce: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:47.795010 10974 tablet_bootstrap.cc:492] T d1d9c30083b746248bc5dfdbb3899496 P e7c0ff7ea89943d8a2cb6de76fc289d9: No bootstrap required, opened a new log
I20260928 14:42:47.795063 10974 ts_tablet_manager.cc:1397] T d1d9c30083b746248bc5dfdbb3899496 P e7c0ff7ea89943d8a2cb6de76fc289d9: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260928 14:42:47.795136 10972 tablet_bootstrap.cc:654] T d1d9c30083b746248bc5dfdbb3899496 P a5e464d260e84647a928ec1b15a5bc71: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:47.795194 10974 raft_consensus.cc:348] T d1d9c30083b746248bc5dfdbb3899496 P e7c0ff7ea89943d8a2cb6de76fc289d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } }
I20260928 14:42:47.795286 10974 raft_consensus.cc:374] T d1d9c30083b746248bc5dfdbb3899496 P e7c0ff7ea89943d8a2cb6de76fc289d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:47.795307 10974 raft_consensus.cc:729] T d1d9c30083b746248bc5dfdbb3899496 P e7c0ff7ea89943d8a2cb6de76fc289d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7c0ff7ea89943d8a2cb6de76fc289d9, State: Initialized, Role: FOLLOWER
I20260928 14:42:47.795368 10974 consensus_queue.cc:260] T d1d9c30083b746248bc5dfdbb3899496 P e7c0ff7ea89943d8a2cb6de76fc289d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } }
I20260928 14:42:47.795497 10974 ts_tablet_manager.cc:1428] T d1d9c30083b746248bc5dfdbb3899496 P e7c0ff7ea89943d8a2cb6de76fc289d9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:47.795521 10988 tablet_replica.cc:439] T d1d9c30083b746248bc5dfdbb3899496 P e7c0ff7ea89943d8a2cb6de76fc289d9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } } }
I20260928 14:42:47.795642 10988 tablet_replica.cc:442] T d1d9c30083b746248bc5dfdbb3899496 P e7c0ff7ea89943d8a2cb6de76fc289d9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:42:47.795889 10972 tablet_bootstrap.cc:492] T d1d9c30083b746248bc5dfdbb3899496 P a5e464d260e84647a928ec1b15a5bc71: No bootstrap required, opened a new log
I20260928 14:42:47.795943 10972 ts_tablet_manager.cc:1397] T d1d9c30083b746248bc5dfdbb3899496 P a5e464d260e84647a928ec1b15a5bc71: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260928 14:42:47.796066 10970 tablet_bootstrap.cc:492] T d1d9c30083b746248bc5dfdbb3899496 P 887d7faee69d4abea170536a8cdde4ce: No bootstrap required, opened a new log
I20260928 14:42:47.796118 10970 ts_tablet_manager.cc:1397] T d1d9c30083b746248bc5dfdbb3899496 P 887d7faee69d4abea170536a8cdde4ce: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260928 14:42:47.796092 10972 raft_consensus.cc:348] T d1d9c30083b746248bc5dfdbb3899496 P a5e464d260e84647a928ec1b15a5bc71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } }
I20260928 14:42:47.796159 10972 raft_consensus.cc:374] T d1d9c30083b746248bc5dfdbb3899496 P a5e464d260e84647a928ec1b15a5bc71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:47.796191 10972 raft_consensus.cc:729] T d1d9c30083b746248bc5dfdbb3899496 P a5e464d260e84647a928ec1b15a5bc71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5e464d260e84647a928ec1b15a5bc71, State: Initialized, Role: FOLLOWER
I20260928 14:42:47.796231 10972 consensus_queue.cc:260] T d1d9c30083b746248bc5dfdbb3899496 P a5e464d260e84647a928ec1b15a5bc71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } }
I20260928 14:42:47.796258 10970 raft_consensus.cc:348] T d1d9c30083b746248bc5dfdbb3899496 P 887d7faee69d4abea170536a8cdde4ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } }
I20260928 14:42:47.796331 10970 raft_consensus.cc:374] T d1d9c30083b746248bc5dfdbb3899496 P 887d7faee69d4abea170536a8cdde4ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:47.796329 10972 ts_tablet_manager.cc:1428] T d1d9c30083b746248bc5dfdbb3899496 P a5e464d260e84647a928ec1b15a5bc71: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:47.796360 10970 raft_consensus.cc:729] T d1d9c30083b746248bc5dfdbb3899496 P 887d7faee69d4abea170536a8cdde4ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 887d7faee69d4abea170536a8cdde4ce, State: Initialized, Role: FOLLOWER
I20260928 14:42:47.796408 10970 consensus_queue.cc:260] T d1d9c30083b746248bc5dfdbb3899496 P 887d7faee69d4abea170536a8cdde4ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } }
I20260928 14:42:47.796496 10970 ts_tablet_manager.cc:1428] T d1d9c30083b746248bc5dfdbb3899496 P 887d7faee69d4abea170536a8cdde4ce: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:47.796448 10977 tablet_replica.cc:439] T d1d9c30083b746248bc5dfdbb3899496 P a5e464d260e84647a928ec1b15a5bc71: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } } }
I20260928 14:42:47.796607 10977 tablet_replica.cc:442] T d1d9c30083b746248bc5dfdbb3899496 P a5e464d260e84647a928ec1b15a5bc71: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:42:47.796576 10973 tablet_replica.cc:439] T d1d9c30083b746248bc5dfdbb3899496 P 887d7faee69d4abea170536a8cdde4ce: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } } }
I20260928 14:42:47.796661 10973 tablet_replica.cc:442] T d1d9c30083b746248bc5dfdbb3899496 P 887d7faee69d4abea170536a8cdde4ce: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:42:47.820848 10988 raft_consensus.cc:482] T d1d9c30083b746248bc5dfdbb3899496 P e7c0ff7ea89943d8a2cb6de76fc289d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:42:47.820940 10988 raft_consensus.cc:504] T d1d9c30083b746248bc5dfdbb3899496 P e7c0ff7ea89943d8a2cb6de76fc289d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } }
I20260928 14:42:47.821223 10988 leader_election.cc:288] T d1d9c30083b746248bc5dfdbb3899496 P e7c0ff7ea89943d8a2cb6de76fc289d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 887d7faee69d4abea170536a8cdde4ce (127.9.33.65:37863), a5e464d260e84647a928ec1b15a5bc71 (127.9.33.67:32995)
I20260928 14:42:47.821512 10901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1d9c30083b746248bc5dfdbb3899496" candidate_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5e464d260e84647a928ec1b15a5bc71" is_pre_election: true
I20260928 14:42:47.821533 10619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1d9c30083b746248bc5dfdbb3899496" candidate_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "887d7faee69d4abea170536a8cdde4ce" is_pre_election: true
I20260928 14:42:47.821619 10901 raft_consensus.cc:2454] T d1d9c30083b746248bc5dfdbb3899496 P a5e464d260e84647a928ec1b15a5bc71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7c0ff7ea89943d8a2cb6de76fc289d9 in term 0.
I20260928 14:42:47.821619 10619 raft_consensus.cc:2454] T d1d9c30083b746248bc5dfdbb3899496 P 887d7faee69d4abea170536a8cdde4ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7c0ff7ea89943d8a2cb6de76fc289d9 in term 0.
I20260928 14:42:47.821772 10695 leader_election.cc:302] T d1d9c30083b746248bc5dfdbb3899496 P e7c0ff7ea89943d8a2cb6de76fc289d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a5e464d260e84647a928ec1b15a5bc71, e7c0ff7ea89943d8a2cb6de76fc289d9; no voters:
I20260928 14:42:47.821869 10988 raft_consensus.cc:2781] T d1d9c30083b746248bc5dfdbb3899496 P e7c0ff7ea89943d8a2cb6de76fc289d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:42:47.821910 10988 raft_consensus.cc:482] T d1d9c30083b746248bc5dfdbb3899496 P e7c0ff7ea89943d8a2cb6de76fc289d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:42:47.821931 10988 raft_consensus.cc:3037] T d1d9c30083b746248bc5dfdbb3899496 P e7c0ff7ea89943d8a2cb6de76fc289d9 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:47.822628 10988 raft_consensus.cc:504] T d1d9c30083b746248bc5dfdbb3899496 P e7c0ff7ea89943d8a2cb6de76fc289d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } }
I20260928 14:42:47.822759 10988 leader_election.cc:288] T d1d9c30083b746248bc5dfdbb3899496 P e7c0ff7ea89943d8a2cb6de76fc289d9 [CANDIDATE]: Term 1 election: Requested vote from peers 887d7faee69d4abea170536a8cdde4ce (127.9.33.65:37863), a5e464d260e84647a928ec1b15a5bc71 (127.9.33.67:32995)
I20260928 14:42:47.822980 10619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1d9c30083b746248bc5dfdbb3899496" candidate_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "887d7faee69d4abea170536a8cdde4ce"
I20260928 14:42:47.822974 10901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1d9c30083b746248bc5dfdbb3899496" candidate_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5e464d260e84647a928ec1b15a5bc71"
I20260928 14:42:47.823060 10901 raft_consensus.cc:3037] T d1d9c30083b746248bc5dfdbb3899496 P a5e464d260e84647a928ec1b15a5bc71 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:47.823060 10619 raft_consensus.cc:3037] T d1d9c30083b746248bc5dfdbb3899496 P 887d7faee69d4abea170536a8cdde4ce [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:47.823823 10619 raft_consensus.cc:2454] T d1d9c30083b746248bc5dfdbb3899496 P 887d7faee69d4abea170536a8cdde4ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7c0ff7ea89943d8a2cb6de76fc289d9 in term 1.
I20260928 14:42:47.823823 10901 raft_consensus.cc:2454] T d1d9c30083b746248bc5dfdbb3899496 P a5e464d260e84647a928ec1b15a5bc71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7c0ff7ea89943d8a2cb6de76fc289d9 in term 1.
I20260928 14:42:47.824038 10695 leader_election.cc:302] T d1d9c30083b746248bc5dfdbb3899496 P e7c0ff7ea89943d8a2cb6de76fc289d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a5e464d260e84647a928ec1b15a5bc71, e7c0ff7ea89943d8a2cb6de76fc289d9; no voters:
I20260928 14:42:47.824160 10988 raft_consensus.cc:2781] T d1d9c30083b746248bc5dfdbb3899496 P e7c0ff7ea89943d8a2cb6de76fc289d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:42:47.824208 10988 raft_consensus.cc:686] T d1d9c30083b746248bc5dfdbb3899496 P e7c0ff7ea89943d8a2cb6de76fc289d9 [term 1 LEADER]: Becoming Leader. State: Replica: e7c0ff7ea89943d8a2cb6de76fc289d9, State: Running, Role: LEADER
I20260928 14:42:47.824266 10988 consensus_queue.cc:237] T d1d9c30083b746248bc5dfdbb3899496 P e7c0ff7ea89943d8a2cb6de76fc289d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } }
I20260928 14:42:47.824599 10978 tablet_replica.cc:439] T d1d9c30083b746248bc5dfdbb3899496 P e7c0ff7ea89943d8a2cb6de76fc289d9: TxnStatusTablet state changed. Reason: New leader e7c0ff7ea89943d8a2cb6de76fc289d9. Latest consensus state: current_term: 1 leader_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } } }
I20260928 14:42:47.824703 10978 tablet_replica.cc:442] T d1d9c30083b746248bc5dfdbb3899496 P e7c0ff7ea89943d8a2cb6de76fc289d9: This TxnStatusTablet replica's current role is: LEADER
I20260928 14:42:47.824872 11013 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260928 14:42:47.824918 10464 catalog_manager.cc:5495] T d1d9c30083b746248bc5dfdbb3899496 P e7c0ff7ea89943d8a2cb6de76fc289d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e7c0ff7ea89943d8a2cb6de76fc289d9 (127.9.33.66). New cstate: current_term: 1 leader_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } health_report { overall_health: UNKNOWN } } }
I20260928 14:42:47.825011 11013 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260928 14:42:47.837088 10967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.834507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60594 (local address 127.9.33.66:45309)
0928 14:42:47.834559 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:47.834561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:47.834572 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:42:47.834671 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:47.834672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:47.834695 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:47.834753 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:42:47.834901 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.835394 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.836314 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.836528 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.836612 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.836790 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:42:47.836829 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:42:47.836899 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:47.836965 (+ 66us) server_negotiation.cc:299] Negotiation successful
0928 14:42:47.836985 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260928 14:42:47.837088 11008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.834435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:45309 (local address 127.0.0.1:60594)
0928 14:42:47.834516 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:47.834529 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:47.834579 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:47.834762 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:47.834762 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:47.834764 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 14:42:47.834832 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:47.834834 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.835482 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.835483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:47.836230 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:47.836233 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.836615 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.836616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:47.836684 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.836705 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 14:42:47.836901 (+ 196us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 14:42:47.836911 (+ 10us) client_negotiation.cc:769] Sending connection context
0928 14:42:47.836953 (+ 42us) client_negotiation.cc:240] Negotiation successful
0928 14:42:47.837001 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260928 14:42:47.838132 10901 raft_consensus.cc:1261] T d1d9c30083b746248bc5dfdbb3899496 P a5e464d260e84647a928ec1b15a5bc71 [term 1 FOLLOWER]: Refusing update from remote peer e7c0ff7ea89943d8a2cb6de76fc289d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:42:47.838132 10619 raft_consensus.cc:1261] T d1d9c30083b746248bc5dfdbb3899496 P 887d7faee69d4abea170536a8cdde4ce [term 1 FOLLOWER]: Refusing update from remote peer e7c0ff7ea89943d8a2cb6de76fc289d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:42:47.838362 10988 consensus_queue.cc:1035] T d1d9c30083b746248bc5dfdbb3899496 P e7c0ff7ea89943d8a2cb6de76fc289d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:42:47.838505 10978 consensus_queue.cc:1035] T d1d9c30083b746248bc5dfdbb3899496 P e7c0ff7ea89943d8a2cb6de76fc289d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:42:47.839078 10977 tablet_replica.cc:439] T d1d9c30083b746248bc5dfdbb3899496 P a5e464d260e84647a928ec1b15a5bc71: TxnStatusTablet state changed. Reason: New leader e7c0ff7ea89943d8a2cb6de76fc289d9. Latest consensus state: current_term: 1 leader_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } } }
I20260928 14:42:47.839182 10977 tablet_replica.cc:442] T d1d9c30083b746248bc5dfdbb3899496 P a5e464d260e84647a928ec1b15a5bc71: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:42:47.839627 10973 tablet_replica.cc:439] T d1d9c30083b746248bc5dfdbb3899496 P 887d7faee69d4abea170536a8cdde4ce: TxnStatusTablet state changed. Reason: New leader e7c0ff7ea89943d8a2cb6de76fc289d9. Latest consensus state: current_term: 1 leader_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } } }
I20260928 14:42:47.839777 10973 tablet_replica.cc:442] T d1d9c30083b746248bc5dfdbb3899496 P 887d7faee69d4abea170536a8cdde4ce: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:42:47.840049 10978 tablet_replica.cc:439] T d1d9c30083b746248bc5dfdbb3899496 P e7c0ff7ea89943d8a2cb6de76fc289d9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } } }
I20260928 14:42:47.840138 10978 tablet_replica.cc:442] T d1d9c30083b746248bc5dfdbb3899496 P e7c0ff7ea89943d8a2cb6de76fc289d9: This TxnStatusTablet replica's current role is: LEADER
I20260928 14:42:47.840114 10980 tablet_replica.cc:439] T d1d9c30083b746248bc5dfdbb3899496 P e7c0ff7ea89943d8a2cb6de76fc289d9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } } }
I20260928 14:42:47.840184 10980 tablet_replica.cc:442] T d1d9c30083b746248bc5dfdbb3899496 P e7c0ff7ea89943d8a2cb6de76fc289d9: This TxnStatusTablet replica's current role is: LEADER
I20260928 14:42:47.840688 10984 mvcc.cc:205] Tried to move back new op lower bound from 7334324501863194624 to 7334324501808701440. Current Snapshot: MvccSnapshot[applied={T|T < 7334324501863194624}]
I20260928 14:42:47.840799 10973 tablet_replica.cc:439] T d1d9c30083b746248bc5dfdbb3899496 P 887d7faee69d4abea170536a8cdde4ce: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } } }
I20260928 14:42:47.840870 10973 tablet_replica.cc:442] T d1d9c30083b746248bc5dfdbb3899496 P 887d7faee69d4abea170536a8cdde4ce: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:42:47.840907 10977 tablet_replica.cc:439] T d1d9c30083b746248bc5dfdbb3899496 P a5e464d260e84647a928ec1b15a5bc71: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "887d7faee69d4abea170536a8cdde4ce" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37863 } } peers { permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 } } peers { permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 } } }
I20260928 14:42:47.840972 10977 tablet_replica.cc:442] T d1d9c30083b746248bc5dfdbb3899496 P a5e464d260e84647a928ec1b15a5bc71: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:42:47.844688 10965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.842281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49944 (local address 127.9.33.65:37863)
0928 14:42:47.842323 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:47.842325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:47.842334 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:42:47.842392 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:47.842393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:47.842413 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:47.842502 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:42:47.842600 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.843090 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.843889 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.844096 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.844171 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.844348 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:42:47.844461 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:42:47.844549 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:47.844588 (+ 39us) server_negotiation.cc:299] Negotiation successful
0928 14:42:47.844610 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 14:42:47.849968 10967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.847433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:54125 (local address 127.9.33.66:45309)
0928 14:42:47.847500 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:47.847502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:47.847606 (+ 104us) server_negotiation.cc:407] Connection header received
0928 14:42:47.847674 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:47.847675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:47.847686 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:47.847802 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:42:47.847998 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.848485 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.849227 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.849426 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.849505 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.849700 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:42:47.849738 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:42:47.849797 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:47.849858 (+ 61us) server_negotiation.cc:299] Negotiation successful
0928 14:42:47.849888 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260928 14:42:47.849996 11020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.847345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:45309 (local address 127.9.33.65:54125)
0928 14:42:47.847529 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:47.847540 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:47.847621 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:47.847808 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:47.847809 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:47.847810 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 14:42:47.847869 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:47.847871 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.848563 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.848564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:47.849137 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:47.849138 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.849522 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.849524 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:47.849591 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.849606 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 14:42:47.849798 (+ 192us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 14:42:47.849801 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:42:47.849853 (+ 52us) client_negotiation.cc:240] Negotiation successful
0928 14:42:47.849899 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":91,"threads_started":1}
I20260928 14:42:47.852486 10901 raft_consensus.cc:1261] T 39dfe83205f84f6da2bfe080d3a908c6 P a5e464d260e84647a928ec1b15a5bc71 [term 1 FOLLOWER]: Refusing update from remote peer 887d7faee69d4abea170536a8cdde4ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:42:47.852629 10761 raft_consensus.cc:1261] T 39dfe83205f84f6da2bfe080d3a908c6 P e7c0ff7ea89943d8a2cb6de76fc289d9 [term 1 FOLLOWER]: Refusing update from remote peer 887d7faee69d4abea170536a8cdde4ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:42:47.852746 11001 consensus_queue.cc:1035] T 39dfe83205f84f6da2bfe080d3a908c6 P 887d7faee69d4abea170536a8cdde4ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5e464d260e84647a928ec1b15a5bc71" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 32995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:42:47.852916 11001 consensus_queue.cc:1035] T 39dfe83205f84f6da2bfe080d3a908c6 P 887d7faee69d4abea170536a8cdde4ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7c0ff7ea89943d8a2cb6de76fc289d9" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:42:47.856828 10983 mvcc.cc:205] Tried to move back new op lower bound from 7334324501921947648 to 7334324501389819904. Current Snapshot: MvccSnapshot[applied={T|T < 7334324501921947648}]
I20260928 14:42:47.863209 11026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.859808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:37863 (local address 127.9.33.66:47159)
0928 14:42:47.859979 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:47.859991 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:47.860057 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:47.860267 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:47.860268 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:47.860269 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 14:42:47.860325 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:47.860327 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.861366 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.861367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:47.862328 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:47.862331 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.862823 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.862824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:47.862882 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.862898 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 14:42:47.863075 (+ 177us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 14:42:47.863077 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 14:42:47.863107 (+ 30us) client_negotiation.cc:240] Negotiation successful
0928 14:42:47.863145 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":103,"threads_started":1}
I20260928 14:42:47.863225 10965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.859875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:47159 (local address 127.9.33.65:37863)
0928 14:42:47.859917 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:47.859920 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:47.860043 (+ 123us) server_negotiation.cc:407] Connection header received
0928 14:42:47.860119 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:47.860120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:47.860132 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:47.860204 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:42:47.860393 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.861236 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.862401 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.862744 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.862823 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.862975 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:42:47.863013 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:42:47.863071 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:47.863122 (+ 51us) server_negotiation.cc:299] Negotiation successful
0928 14:42:47.863145 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 14:42:47.872725 10463 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:42:47.872850 10463 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:42:47.873486 10463 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 990c5809ef95459b882bfeb2232d16b5: Sending DeleteTablet for 3 replicas of tablet 39dfe83205f84f6da2bfe080d3a908c6
I20260928 14:42:47.873744 10601 tablet_service.cc:1514] Processing DeleteTablet for tablet 39dfe83205f84f6da2bfe080d3a908c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:42:47 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:49936
I20260928 14:42:47.873787 10881 tablet_service.cc:1514] Processing DeleteTablet for tablet 39dfe83205f84f6da2bfe080d3a908c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:42:47 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:54704
I20260928 14:42:47.873879 10994 tablet_replica.cc:330] T 39dfe83205f84f6da2bfe080d3a908c6 P a5e464d260e84647a928ec1b15a5bc71: stopping tablet replica
I20260928 14:42:47.873857 10993 tablet_replica.cc:330] T 39dfe83205f84f6da2bfe080d3a908c6 P 887d7faee69d4abea170536a8cdde4ce: stopping tablet replica
I20260928 14:42:47.873925 10994 raft_consensus.cc:2229] T 39dfe83205f84f6da2bfe080d3a908c6 P a5e464d260e84647a928ec1b15a5bc71 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:42:47.873929 10993 raft_consensus.cc:2229] T 39dfe83205f84f6da2bfe080d3a908c6 P 887d7faee69d4abea170536a8cdde4ce [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:42:47.873961 10994 raft_consensus.cc:2258] T 39dfe83205f84f6da2bfe080d3a908c6 P a5e464d260e84647a928ec1b15a5bc71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:42:47.874022 10993 raft_consensus.cc:2258] T 39dfe83205f84f6da2bfe080d3a908c6 P 887d7faee69d4abea170536a8cdde4ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:42:47.874224 10741 tablet_service.cc:1514] Processing DeleteTablet for tablet 39dfe83205f84f6da2bfe080d3a908c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:42:47 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:60576
I20260928 14:42:47.874360 10993 ts_tablet_manager.cc:1905] T 39dfe83205f84f6da2bfe080d3a908c6 P 887d7faee69d4abea170536a8cdde4ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:42:47.874423 10994 ts_tablet_manager.cc:1905] T 39dfe83205f84f6da2bfe080d3a908c6 P a5e464d260e84647a928ec1b15a5bc71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:42:47.876441 10993 ts_tablet_manager.cc:1918] T 39dfe83205f84f6da2bfe080d3a908c6 P 887d7faee69d4abea170536a8cdde4ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:42:47.876508 10993 log.cc:1196] T 39dfe83205f84f6da2bfe080d3a908c6 P 887d7faee69d4abea170536a8cdde4ce: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-0/wal/wals/39dfe83205f84f6da2bfe080d3a908c6
I20260928 14:42:47.876745 10993 ts_tablet_manager.cc:1939] T 39dfe83205f84f6da2bfe080d3a908c6 P 887d7faee69d4abea170536a8cdde4ce: Deleting consensus metadata
I20260928 14:42:47.876892 10995 tablet_replica.cc:330] T 39dfe83205f84f6da2bfe080d3a908c6 P e7c0ff7ea89943d8a2cb6de76fc289d9: stopping tablet replica
I20260928 14:42:47.876988 10995 raft_consensus.cc:2229] T 39dfe83205f84f6da2bfe080d3a908c6 P e7c0ff7ea89943d8a2cb6de76fc289d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:42:47.877035 10995 raft_consensus.cc:2258] T 39dfe83205f84f6da2bfe080d3a908c6 P e7c0ff7ea89943d8a2cb6de76fc289d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:42:47.877172 10451 catalog_manager.cc:4841] TS 887d7faee69d4abea170536a8cdde4ce (127.9.33.65:37863): tablet 39dfe83205f84f6da2bfe080d3a908c6 (table test-table [id=315791256d584fa081a2d2d793896e1d]) successfully deleted
I20260928 14:42:47.877503 10995 ts_tablet_manager.cc:1905] T 39dfe83205f84f6da2bfe080d3a908c6 P e7c0ff7ea89943d8a2cb6de76fc289d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:42:47.879165 10995 ts_tablet_manager.cc:1918] T 39dfe83205f84f6da2bfe080d3a908c6 P e7c0ff7ea89943d8a2cb6de76fc289d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:42:47.879243 10995 log.cc:1196] T 39dfe83205f84f6da2bfe080d3a908c6 P e7c0ff7ea89943d8a2cb6de76fc289d9: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-1/wal/wals/39dfe83205f84f6da2bfe080d3a908c6
I20260928 14:42:47.879482 10995 ts_tablet_manager.cc:1939] T 39dfe83205f84f6da2bfe080d3a908c6 P e7c0ff7ea89943d8a2cb6de76fc289d9: Deleting consensus metadata
I20260928 14:42:47.879868 10994 ts_tablet_manager.cc:1918] T 39dfe83205f84f6da2bfe080d3a908c6 P a5e464d260e84647a928ec1b15a5bc71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:42:47.879899 10451 catalog_manager.cc:4841] TS e7c0ff7ea89943d8a2cb6de76fc289d9 (127.9.33.66:45309): tablet 39dfe83205f84f6da2bfe080d3a908c6 (table test-table [id=315791256d584fa081a2d2d793896e1d]) successfully deleted
I20260928 14:42:47.879930 10994 log.cc:1196] T 39dfe83205f84f6da2bfe080d3a908c6 P a5e464d260e84647a928ec1b15a5bc71: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790606561545179-9349-0/minicluster-data/ts-2/wal/wals/39dfe83205f84f6da2bfe080d3a908c6
I20260928 14:42:47.880147 10994 ts_tablet_manager.cc:1939] T 39dfe83205f84f6da2bfe080d3a908c6 P a5e464d260e84647a928ec1b15a5bc71: Deleting consensus metadata
I20260928 14:42:47.880511 10451 catalog_manager.cc:4841] TS a5e464d260e84647a928ec1b15a5bc71 (127.9.33.67:32995): tablet 39dfe83205f84f6da2bfe080d3a908c6 (table test-table [id=315791256d584fa081a2d2d793896e1d]) successfully deleted
Sep 28 14:42:47 dist-test-slave-pv0w krb5kdc[10419](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606567, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.33.65@KRBTEST.COM
I20260928 14:42:47.900771 10965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.876523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49948 (local address 127.9.33.65:37863)
0928 14:42:47.876570 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:47.876572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:47.891965 (+ 15393us) server_negotiation.cc:407] Connection header received
0928 14:42:47.892047 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:47.892048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:47.892059 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:47.892140 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:47.892757 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.893255 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.893998 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.894192 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.894280 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.897306 (+ 3026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:47.897317 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:47.897317 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:47.897325 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:47.899272 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:47.899740 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:47.899744 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:47.899745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:47.899810 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:47.900038 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:47.900040 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:47.900040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:47.900211 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:47.900330 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:47.900562 (+ 232us) server_negotiation.cc:299] Negotiation successful
0928 14:42:47.900651 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260928 14:42:47.902649 10650 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49948
I20260928 14:42:47.909950 10514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:47.903606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49240 (local address 127.9.33.126:37901)
0928 14:42:47.903652 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:47.903654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:47.903886 (+ 232us) server_negotiation.cc:407] Connection header received
0928 14:42:47.903949 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:47.903950 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:47.903960 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:47.904005 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:47.904516 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.905080 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.905771 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:47.905989 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:47.906070 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:47.907301 (+ 1231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:47.907303 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:47.907303 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:47.907310 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:47.908742 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:47.909126 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:47.909128 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:47.909129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:47.909187 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:47.909393 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:47.909394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:47.909394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:47.909555 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:47.909661 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:47.909765 (+ 104us) server_negotiation.cc:299] Negotiation successful
0928 14:42:47.909844 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260928 14:42:47.910286 10463 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49240
I20260928 14:42:47.910844 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 10536
I20260928 14:42:47.918715 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 10676
I20260928 14:42:47.924866 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 10816
I20260928 14:42:47.929919 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 10432
2026-09-28T14:42:47Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2345 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:42:47 dist-test-slave-pv0w krb5kdc[11035](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:42:47 dist-test-slave-pv0w krb5kdc[11035](info): set up 2 sockets
Sep 28 14:42:47 dist-test-slave-pv0w krb5kdc[11035](info): commencing operation
krb5kdc: starting...
W20260928 14:42:48.968152 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:42:49 dist-test-slave-pv0w krb5kdc[11035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:42:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:42:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.33.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
I20260928 14:42:49.086061 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:46127
--webserver_interface=127.9.33.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:45303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:46127
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:49.168259 11048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:49.168473 11048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:49.168495 11048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:49.169586 11048 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:42:49.169620 11048 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260928 14:42:49.169633 11048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:49.169646 11048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:49.169658 11048 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:42:49.169670 11048 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:42:49.171028 11048 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:45303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:46127
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:46127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:49.171212 11048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:49.171387 11048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:49.176662 11054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:49.176712 11053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:49.176662 11056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:49.176680 11048 server_base.cc:1034] running on GCE node
I20260928 14:42:49.177193 11048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:49.177565 11048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:49.178730 11048 hybrid_clock.cc:648] HybridClock initialized: now 1790606569178710 us; error 72 us; skew 500 ppm
Sep 28 14:42:49 dist-test-slave-pv0w krb5kdc[11035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606569, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:49.181317 11048 init.cc:377] Logged in from keytab as kudu/127.9.33.126@KRBTEST.COM (short username kudu)
I20260928 14:42:49.181777 11048 webserver.cc:460] Webserver started at http://127.9.33.126:32857/ using document root <none> and password file <none>
I20260928 14:42:49.182013 11048 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:49.182083 11048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:49.182236 11048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:49.183435 11048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "4e38cb65782d4298acda01cec350da9f"
format_stamp: "Formatted at 2026-09-28 14:42:49 on dist-test-slave-pv0w"
server_key: "921d34577abaca5cf1c2c3333ee6515a"
server_key_iv: "b91ea72b5c4d18507ea4c5f803d7ef0f"
server_key_version: "encryptionkey@0"
I20260928 14:42:49.183871 11048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "4e38cb65782d4298acda01cec350da9f"
format_stamp: "Formatted at 2026-09-28 14:42:49 on dist-test-slave-pv0w"
server_key: "921d34577abaca5cf1c2c3333ee6515a"
server_key_iv: "b91ea72b5c4d18507ea4c5f803d7ef0f"
server_key_version: "encryptionkey@0"
I20260928 14:42:49.185225 11048 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:42:49.185982 11063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:49.186180 11048 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 14:42:49.186281 11048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "4e38cb65782d4298acda01cec350da9f"
format_stamp: "Formatted at 2026-09-28 14:42:49 on dist-test-slave-pv0w"
server_key: "921d34577abaca5cf1c2c3333ee6515a"
server_key_iv: "b91ea72b5c4d18507ea4c5f803d7ef0f"
server_key_version: "encryptionkey@0"
I20260928 14:42:49.186363 11048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:49.218788 11048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:49.221385 11048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:49.221560 11048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:49.226411 11048 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:46127
I20260928 14:42:49.226459 11125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:46127 every 8 connection(s)
I20260928 14:42:49.226804 11048 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:42:49.227481 11126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:49.229794 11126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e38cb65782d4298acda01cec350da9f: Bootstrap starting.
I20260928 14:42:49.230360 11126 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e38cb65782d4298acda01cec350da9f: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:49.230610 11126 log.cc:824] T 00000000000000000000000000000000 P 4e38cb65782d4298acda01cec350da9f: Log is configured to *not* fsync() on all Append() calls
I20260928 14:42:49.231293 11126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e38cb65782d4298acda01cec350da9f: No bootstrap required, opened a new log
I20260928 14:42:49.231381 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 11048
I20260928 14:42:49.231499 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:42:49.231668 9349 external_mini_cluster.cc:1428] Setting key b8371e7d5090e076dbe8e91914cc7b70
I20260928 14:42:49.234131 11126 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e38cb65782d4298acda01cec350da9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e38cb65782d4298acda01cec350da9f" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 46127 } }
I20260928 14:42:49.234321 11126 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e38cb65782d4298acda01cec350da9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:49.234364 11126 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e38cb65782d4298acda01cec350da9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e38cb65782d4298acda01cec350da9f, State: Initialized, Role: FOLLOWER
I20260928 14:42:49.234496 11126 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e38cb65782d4298acda01cec350da9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e38cb65782d4298acda01cec350da9f" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 46127 } }
I20260928 14:42:49.234579 11126 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e38cb65782d4298acda01cec350da9f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:42:49.234603 11126 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e38cb65782d4298acda01cec350da9f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:42:49.234668 11126 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e38cb65782d4298acda01cec350da9f [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:49.235349 11126 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e38cb65782d4298acda01cec350da9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e38cb65782d4298acda01cec350da9f" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 46127 } }
I20260928 14:42:49.235462 11126 leader_election.cc:302] T 00000000000000000000000000000000 P 4e38cb65782d4298acda01cec350da9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e38cb65782d4298acda01cec350da9f; no voters:
I20260928 14:42:49.235630 11126 leader_election.cc:288] T 00000000000000000000000000000000 P 4e38cb65782d4298acda01cec350da9f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:42:49.235720 11131 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e38cb65782d4298acda01cec350da9f [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:42:49.235927 11131 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e38cb65782d4298acda01cec350da9f [term 1 LEADER]: Becoming Leader. State: Replica: 4e38cb65782d4298acda01cec350da9f, State: Running, Role: LEADER
I20260928 14:42:49.235929 11126 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e38cb65782d4298acda01cec350da9f [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:42:49.236029 11131 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e38cb65782d4298acda01cec350da9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e38cb65782d4298acda01cec350da9f" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 46127 } }
I20260928 14:42:49.237630 11132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e38cb65782d4298acda01cec350da9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e38cb65782d4298acda01cec350da9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e38cb65782d4298acda01cec350da9f" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 46127 } } }
I20260928 14:42:49.237674 11133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e38cb65782d4298acda01cec350da9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e38cb65782d4298acda01cec350da9f. Latest consensus state: current_term: 1 leader_uuid: "4e38cb65782d4298acda01cec350da9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e38cb65782d4298acda01cec350da9f" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 46127 } } }
I20260928 14:42:49.237816 11133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e38cb65782d4298acda01cec350da9f [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:49.238049 11132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e38cb65782d4298acda01cec350da9f [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:49.238058 11141 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:42:49.238629 11141 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:42:49.240404 11141 catalog_manager.cc:1347] Generated new cluster ID: d99b3232fa94473d8860c9850450f2ad
I20260928 14:42:49.240456 11141 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 14:42:49 dist-test-slave-pv0w krb5kdc[11035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:42:49.244407 11130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:49.232245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46810 (local address 127.9.33.126:46127)
0928 14:42:49.232457 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:49.232467 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:49.232502 (+ 35us) server_negotiation.cc:407] Connection header received
0928 14:42:49.234078 (+ 1576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:49.234089 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:49.234157 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:49.234320 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:49.235140 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.236778 (+ 1638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.237611 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.238282 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.238410 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:49.241159 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:49.241163 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:49.241163 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:49.241174 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:49.242900 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:49.243251 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:49.243253 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:49.243254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:49.243353 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:49.243535 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:49.243536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:49.243536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:49.243833 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:49.243933 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:49.244065 (+ 132us) server_negotiation.cc:299] Negotiation successful
0928 14:42:49.244167 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":88,"threads_started":1}
I20260928 14:42:49.278522 11141 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:42:49.279121 11141 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:42:49.283399 11141 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e38cb65782d4298acda01cec350da9f: Generated new TSK 0
I20260928 14:42:49.283609 11141 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.33.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab.
Entry for principal kudu/127.9.33.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab.
I20260928 14:42:49.314558 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.65:0
--local_ip_for_outbound_sockets=127.9.33.65
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:46127
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:45303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:49.418421 11152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:49.418641 11152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:49.418668 11152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:49.419977 11152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:49.420019 11152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:49.420042 11152 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260928 14:42:49.420077 11152 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 14:42:49.420114 11152 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.65
I20260928 14:42:49.421607 11152 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:45303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.33.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.33.126:46127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.65
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:49.421866 11152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:49.422119 11152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:49.427430 11157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:49.427425 11152 server_base.cc:1034] running on GCE node
W20260928 14:42:49.427402 11158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:49.427408 11160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:49.427850 11152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:49.428073 11152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:49.429224 11152 hybrid_clock.cc:648] HybridClock initialized: now 1790606569429213 us; error 27 us; skew 500 ppm
Sep 28 14:42:49 dist-test-slave-pv0w krb5kdc[11035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606569, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:49.431906 11152 init.cc:377] Logged in from keytab as kudu/127.9.33.65@KRBTEST.COM (short username kudu)
I20260928 14:42:49.432377 11152 webserver.cc:460] Webserver started at http://127.9.33.65:39919/ using document root <none> and password file <none>
I20260928 14:42:49.432633 11152 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:49.432682 11152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:49.432854 11152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:49.434201 11152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-0/data/instance:
uuid: "6475771aee444d78832d41ed481d4255"
format_stamp: "Formatted at 2026-09-28 14:42:49 on dist-test-slave-pv0w"
server_key: "079356ff6331af2f13da9486a655e667"
server_key_iv: "93dc99a4de5f4b62fba46511d3a013f4"
server_key_version: "encryptionkey@0"
I20260928 14:42:49.434623 11152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance:
uuid: "6475771aee444d78832d41ed481d4255"
format_stamp: "Formatted at 2026-09-28 14:42:49 on dist-test-slave-pv0w"
server_key: "079356ff6331af2f13da9486a655e667"
server_key_iv: "93dc99a4de5f4b62fba46511d3a013f4"
server_key_version: "encryptionkey@0"
I20260928 14:42:49.436295 11152 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260928 14:42:49.437254 11167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:49.437413 11152 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:42:49.437501 11152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-0/wal
uuid: "6475771aee444d78832d41ed481d4255"
format_stamp: "Formatted at 2026-09-28 14:42:49 on dist-test-slave-pv0w"
server_key: "079356ff6331af2f13da9486a655e667"
server_key_iv: "93dc99a4de5f4b62fba46511d3a013f4"
server_key_version: "encryptionkey@0"
I20260928 14:42:49.437561 11152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:49.455086 11152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:49.457144 11152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:49.457262 11152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:49.457814 11152 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:42:49.457857 11152 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:49.457881 11152 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:42:49.457894 11152 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 14:42:49 dist-test-slave-pv0w krb5kdc[11035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606569, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:42:49.465247 11152 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.65:46725
I20260928 14:42:49.465322 11282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.65:46725 every 8 connection(s)
I20260928 14:42:49.465626 11152 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
I20260928 14:42:49.468129 11130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:49.458260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:54891 (local address 127.9.33.126:46127)
0928 14:42:49.458341 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:49.458344 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:49.458526 (+ 182us) server_negotiation.cc:407] Connection header received
0928 14:42:49.459059 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:49.459061 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:49.459075 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:49.459164 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:49.459944 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.460845 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.461968 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.462219 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.462303 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:49.465391 (+ 3088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:49.465394 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:49.465394 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:49.465404 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:49.466913 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:49.467284 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:49.467285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:49.467286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:49.467335 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:49.467533 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:49.467534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:49.467534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:49.467635 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:49.467684 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:49.467996 (+ 312us) server_negotiation.cc:299] Negotiation successful
0928 14:42:49.468065 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260928 14:42:49.468267 11176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:49.458232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46127 (local address 127.9.33.65:54891)
0928 14:42:49.458393 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:49.458436 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:49.458961 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:49.459254 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:49.459259 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:49.459568 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:49.459820 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:49.459825 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.460958 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.460959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:49.461856 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:49.461859 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.462323 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.462324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:49.462478 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:49.462989 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:49.463000 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:49.465267 (+ 2267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:49.466995 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:49.467000 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:49.467002 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:49.467217 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:49.467399 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:49.467399 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:49.467400 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:49.467482 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:49.467693 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:49.467698 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:49.467878 (+ 180us) client_negotiation.cc:769] Sending connection context
0928 14:42:49.467964 (+ 86us) client_negotiation.cc:240] Negotiation successful
0928 14:42:49.468084 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":89,"threads_started":1}
I20260928 14:42:49.468842 11283 heartbeater.cc:347] Connected to a master server at 127.9.33.126:46127
I20260928 14:42:49.468915 11283 heartbeater.cc:464] Registering TS with master...
I20260928 14:42:49.469096 11283 heartbeater.cc:511] Master 127.9.33.126:46127 requested a full tablet report, sending...
I20260928 14:42:49.469573 11079 ts_manager.cc:194] Registered new tserver with Master: 6475771aee444d78832d41ed481d4255 (127.9.33.65:46725)
I20260928 14:42:49.470436 11079 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.65@KRBTEST.COM'} at 127.9.33.65:54891
I20260928 14:42:49.470631 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 11152
I20260928 14:42:49.470715 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance
I20260928 14:42:49.470916 9349 external_mini_cluster.cc:1428] Setting key 2db97cd5491b850539f0beac8c7fcc4d
I20260928 14:42:49.475939 11130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:49.469964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:55941 (local address 127.9.33.126:46127)
0928 14:42:49.470004 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:49.470006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:49.470122 (+ 116us) server_negotiation.cc:407] Connection header received
0928 14:42:49.470157 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:49.470158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:49.470168 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:49.470220 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:49.470527 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.471069 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.471934 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.472140 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.472226 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:49.473320 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:49.473322 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:49.473322 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:49.473328 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:49.474791 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:49.475198 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:49.475199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:49.475199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:49.475246 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:49.475445 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:49.475445 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:49.475446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:49.475532 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:49.475581 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:49.475816 (+ 235us) server_negotiation.cc:299] Negotiation successful
0928 14:42:49.475880 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 14:42:49.476043 11289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:49.469851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46127 (local address 127.9.33.65:55941)
0928 14:42:49.470040 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:49.470051 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:49.470112 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:49.470224 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:49.470224 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:49.470374 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:49.470429 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:49.470430 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.471166 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.471167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:49.471845 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:49.471847 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.472227 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.472228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:49.472301 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:49.472737 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:49.472741 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:49.473210 (+ 469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:49.474879 (+ 1669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:49.474880 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:49.474881 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:49.475126 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:49.475311 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:49.475312 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:49.475313 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:49.475386 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:49.475599 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:49.475600 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:49.475668 (+ 68us) client_negotiation.cc:769] Sending connection context
0928 14:42:49.475760 (+ 92us) client_negotiation.cc:240] Negotiation successful
0928 14:42:49.475880 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":34,"thread_start_us":78,"threads_started":1}
WARNING: no policy specified for kudu/127.9.33.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab.
Entry for principal kudu/127.9.33.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab.
I20260928 14:42:49.498026 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.66:0
--local_ip_for_outbound_sockets=127.9.33.66
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:46127
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:45303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:49.586442 11292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:49.586660 11292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:49.586688 11292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:49.587980 11292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:49.588018 11292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:49.588042 11292 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260928 14:42:49.588054 11292 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 14:42:49.588073 11292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.66
I20260928 14:42:49.589718 11292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:45303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.33.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.33.126:46127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.66
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:49.589911 11292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:49.590155 11292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:49.595623 11297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:49.595634 11298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:49.595921 11300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:49.595983 11292 server_base.cc:1034] running on GCE node
I20260928 14:42:49.596203 11292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:49.596464 11292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:49.597721 11292 hybrid_clock.cc:648] HybridClock initialized: now 1790606569597704 us; error 33 us; skew 500 ppm
Sep 28 14:42:49 dist-test-slave-pv0w krb5kdc[11035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606569, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:49.600565 11292 init.cc:377] Logged in from keytab as kudu/127.9.33.66@KRBTEST.COM (short username kudu)
I20260928 14:42:49.601071 11292 webserver.cc:460] Webserver started at http://127.9.33.66:41799/ using document root <none> and password file <none>
I20260928 14:42:49.601318 11292 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:49.601387 11292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:49.601542 11292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:49.602602 11292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-1/data/instance:
uuid: "8f0bbb4eaedd49b9b5a0d487b3860300"
format_stamp: "Formatted at 2026-09-28 14:42:49 on dist-test-slave-pv0w"
server_key: "a4c92d4d6d0773ca361b1967fc385ebf"
server_key_iv: "4b67be2f2faa50fd2616ab80f3637390"
server_key_version: "encryptionkey@0"
I20260928 14:42:49.602979 11292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance:
uuid: "8f0bbb4eaedd49b9b5a0d487b3860300"
format_stamp: "Formatted at 2026-09-28 14:42:49 on dist-test-slave-pv0w"
server_key: "a4c92d4d6d0773ca361b1967fc385ebf"
server_key_iv: "4b67be2f2faa50fd2616ab80f3637390"
server_key_version: "encryptionkey@0"
I20260928 14:42:49.604418 11292 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260928 14:42:49.605185 11307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:49.605365 11292 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:42:49.605469 11292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-1/wal
uuid: "8f0bbb4eaedd49b9b5a0d487b3860300"
format_stamp: "Formatted at 2026-09-28 14:42:49 on dist-test-slave-pv0w"
server_key: "a4c92d4d6d0773ca361b1967fc385ebf"
server_key_iv: "4b67be2f2faa50fd2616ab80f3637390"
server_key_version: "encryptionkey@0"
I20260928 14:42:49.605551 11292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:49.621230 11292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:49.623225 11292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:49.623397 11292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:49.624046 11292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:42:49.624118 11292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:49.624151 11292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:42:49.624168 11292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 14:42:49 dist-test-slave-pv0w krb5kdc[11035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606569, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:42:49.631487 11292 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.66:36645
I20260928 14:42:49.631852 11422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.66:36645 every 8 connection(s)
I20260928 14:42:49.631950 11292 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
I20260928 14:42:49.633318 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 11292
I20260928 14:42:49.633411 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance
I20260928 14:42:49.633603 9349 external_mini_cluster.cc:1428] Setting key 8ee30767472d59e01c31334dd6127495
I20260928 14:42:49.633882 11130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:49.624384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:55063 (local address 127.9.33.126:46127)
0928 14:42:49.624429 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:49.624431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:49.624676 (+ 245us) server_negotiation.cc:407] Connection header received
0928 14:42:49.625216 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:49.625217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:49.625232 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:49.625283 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:49.625895 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.626418 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.627548 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.627901 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.627992 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:49.630640 (+ 2648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:49.630642 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:49.630642 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:49.630650 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:49.632348 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:49.632787 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:49.632789 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:49.632789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:49.632858 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:49.633121 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:49.633122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:49.633122 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:49.633278 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:49.633351 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:49.633685 (+ 334us) server_negotiation.cc:299] Negotiation successful
0928 14:42:49.633773 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 14:42:49.634119 11316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:49.624380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46127 (local address 127.9.33.66:55063)
0928 14:42:49.624561 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:49.624597 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:49.625100 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:49.625355 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:49.625364 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:49.625606 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:49.625790 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:49.625794 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.626517 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.626518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:49.627384 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:49.627386 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.628000 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.628001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:49.628141 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:49.628675 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:49.628684 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:49.630535 (+ 1851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:49.632432 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:49.632437 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:49.632439 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:49.632706 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:49.632931 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:49.632932 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:49.632932 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:49.633041 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:49.633356 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:49.633360 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:49.633538 (+ 178us) client_negotiation.cc:769] Sending connection context
0928 14:42:49.633635 (+ 97us) client_negotiation.cc:240] Negotiation successful
0928 14:42:49.633861 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":107,"threads_started":1}
I20260928 14:42:49.634599 11423 heartbeater.cc:347] Connected to a master server at 127.9.33.126:46127
I20260928 14:42:49.634745 11423 heartbeater.cc:464] Registering TS with master...
I20260928 14:42:49.634999 11423 heartbeater.cc:511] Master 127.9.33.126:46127 requested a full tablet report, sending...
I20260928 14:42:49.635435 11080 ts_manager.cc:194] Registered new tserver with Master: 8f0bbb4eaedd49b9b5a0d487b3860300 (127.9.33.66:36645)
I20260928 14:42:49.636041 11080 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.66@KRBTEST.COM'} at 127.9.33.66:55063
WARNING: no policy specified for kudu/127.9.33.67@KRBTEST.COM; defaulting to no policy
I20260928 14:42:49.644449 11430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:49.635817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46127 (local address 127.9.33.66:33285)
0928 14:42:49.635956 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:49.635967 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:49.636031 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:49.636188 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:49.636189 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:49.636440 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:49.636519 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:49.636521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.639897 (+ 3376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.639898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:49.640435 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:49.640436 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.640981 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.640982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:49.641050 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:49.641605 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:49.641611 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:49.641972 (+ 361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:49.643510 (+ 1538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:49.643511 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:49.643511 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:49.643682 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:49.643917 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:49.643918 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:49.643918 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:49.643973 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:49.644201 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:49.644202 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:49.644240 (+ 38us) client_negotiation.cc:769] Sending connection context
0928 14:42:49.644289 (+ 49us) client_negotiation.cc:240] Negotiation successful
0928 14:42:49.644373 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":89,"threads_started":1}
I20260928 14:42:49.644534 11130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:49.635852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:33285 (local address 127.9.33.126:46127)
0928 14:42:49.635910 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:49.635912 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:49.636040 (+ 128us) server_negotiation.cc:407] Connection header received
0928 14:42:49.636072 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:49.636073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:49.636085 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:49.636143 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:49.636609 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.637170 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.640554 (+ 3384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.640881 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.640966 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:49.642067 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:49.642069 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:49.642070 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:49.642078 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:49.643436 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:49.643793 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:49.643794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:49.643794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:49.643857 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:49.644020 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:49.644021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:49.644021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:49.644150 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:49.644197 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:49.644306 (+ 109us) server_negotiation.cc:299] Negotiation successful
0928 14:42:49.644361 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab.
Entry for principal kudu/127.9.33.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab.
I20260928 14:42:49.660761 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.67:0
--local_ip_for_outbound_sockets=127.9.33.67
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:46127
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:45303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:49.742923 11432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:49.743144 11432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:49.743170 11432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:49.744350 11432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:49.744419 11432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:49.744457 11432 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260928 14:42:49.744505 11432 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 14:42:49.744545 11432 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.67
I20260928 14:42:49.746047 11432 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:45303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.33.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.33.126:46127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.67
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:49.746345 11432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:49.746585 11432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:49.752600 11437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:49.752641 11438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:49.752856 11440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:49.753137 11432 server_base.cc:1034] running on GCE node
I20260928 14:42:49.753383 11432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:49.753698 11432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:49.754914 11432 hybrid_clock.cc:648] HybridClock initialized: now 1790606569754884 us; error 40 us; skew 500 ppm
Sep 28 14:42:49 dist-test-slave-pv0w krb5kdc[11035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606569, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:49.757617 11432 init.cc:377] Logged in from keytab as kudu/127.9.33.67@KRBTEST.COM (short username kudu)
I20260928 14:42:49.758239 11432 webserver.cc:460] Webserver started at http://127.9.33.67:35283/ using document root <none> and password file <none>
I20260928 14:42:49.758512 11432 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:49.758591 11432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:49.758770 11432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:49.760054 11432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-2/data/instance:
uuid: "4cfa07b117ef40fca06be7d703b98707"
format_stamp: "Formatted at 2026-09-28 14:42:49 on dist-test-slave-pv0w"
server_key: "24e40ac9525067a1676c7475d40fa9ab"
server_key_iv: "462fefbaaf908d53c588c019e92526ba"
server_key_version: "encryptionkey@0"
I20260928 14:42:49.760456 11432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance:
uuid: "4cfa07b117ef40fca06be7d703b98707"
format_stamp: "Formatted at 2026-09-28 14:42:49 on dist-test-slave-pv0w"
server_key: "24e40ac9525067a1676c7475d40fa9ab"
server_key_iv: "462fefbaaf908d53c588c019e92526ba"
server_key_version: "encryptionkey@0"
I20260928 14:42:49.761920 11432 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:42:49.762648 11447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:49.762823 11432 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:42:49.762933 11432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-2/wal
uuid: "4cfa07b117ef40fca06be7d703b98707"
format_stamp: "Formatted at 2026-09-28 14:42:49 on dist-test-slave-pv0w"
server_key: "24e40ac9525067a1676c7475d40fa9ab"
server_key_iv: "462fefbaaf908d53c588c019e92526ba"
server_key_version: "encryptionkey@0"
I20260928 14:42:49.763011 11432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:49.787117 11432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:49.789222 11432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:49.789382 11432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:49.790032 11432 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:42:49.790091 11432 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:49.790129 11432 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:42:49.790153 11432 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 14:42:49 dist-test-slave-pv0w krb5kdc[11035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606569, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:42:49.797778 11432 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.67:39601
I20260928 14:42:49.798226 11432 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
I20260928 14:42:49.797863 11562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.67:39601 every 8 connection(s)
I20260928 14:42:49.800453 11130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:49.790445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:33597 (local address 127.9.33.126:46127)
0928 14:42:49.790517 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:49.790519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:49.790774 (+ 255us) server_negotiation.cc:407] Connection header received
0928 14:42:49.791273 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:49.791274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:49.791287 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:49.791375 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:49.792103 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.792948 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.794013 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.794364 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.794447 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:49.797267 (+ 2820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:49.797269 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:49.797269 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:49.797277 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:49.799092 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:49.799531 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:49.799533 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:49.799534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:49.799595 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:49.799911 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:49.799912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:49.799912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:49.800028 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:49.800087 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:49.800314 (+ 227us) server_negotiation.cc:299] Negotiation successful
0928 14:42:49.800390 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260928 14:42:49.800680 11456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:49.790456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46127 (local address 127.9.33.67:33597)
0928 14:42:49.790634 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:49.790676 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:49.791165 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:49.791430 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:49.791433 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:49.791818 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:49.792024 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:49.792029 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.793078 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.793079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:49.793921 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:49.793923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.794462 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.794463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:49.794617 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:49.795252 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:49.795264 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:49.797189 (+ 1925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:49.799186 (+ 1997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:49.799192 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:49.799194 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:49.799440 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:49.799665 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:49.799666 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:49.799666 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:49.799837 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:49.800091 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:49.800096 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:49.800214 (+ 118us) client_negotiation.cc:769] Sending connection context
0928 14:42:49.800292 (+ 78us) client_negotiation.cc:240] Negotiation successful
0928 14:42:49.800430 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":88,"threads_started":1}
I20260928 14:42:49.801131 11563 heartbeater.cc:347] Connected to a master server at 127.9.33.126:46127
I20260928 14:42:49.801218 11563 heartbeater.cc:464] Registering TS with master...
I20260928 14:42:49.801436 11563 heartbeater.cc:511] Master 127.9.33.126:46127 requested a full tablet report, sending...
I20260928 14:42:49.801775 11079 ts_manager.cc:194] Registered new tserver with Master: 4cfa07b117ef40fca06be7d703b98707 (127.9.33.67:39601)
I20260928 14:42:49.802114 11079 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.67@KRBTEST.COM'} at 127.9.33.67:33597
I20260928 14:42:49.805889 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 11432
I20260928 14:42:49.805961 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance
I20260928 14:42:49.806176 9349 external_mini_cluster.cc:1428] Setting key 0ece20e3787a4d8b4d465e5ffe258381
I20260928 14:42:49.807020 9349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:42:49.808241 11130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:49.802317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:49365 (local address 127.9.33.126:46127)
0928 14:42:49.802474 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:49.802476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:49.802486 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:42:49.802507 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:49.802508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:49.802518 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:49.802584 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:49.802863 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.803383 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.804161 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.804370 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.804434 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:49.805266 (+ 832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:49.805267 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:49.805268 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:49.805274 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:49.807086 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:49.807469 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:49.807470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:49.807470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:49.807526 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:49.807775 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:49.807776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:49.807777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:49.807892 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:49.807945 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:49.808119 (+ 174us) server_negotiation.cc:299] Negotiation successful
0928 14:42:49.808181 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260928 14:42:49.808398 11569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:49.802244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46127 (local address 127.9.33.67:49365)
0928 14:42:49.802403 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:49.802414 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:49.802468 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:49.802587 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:49.802588 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:49.802736 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:49.802790 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:49.802791 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.803474 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.803474 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:49.804084 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:49.804086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.804441 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.804441 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:49.804495 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:49.804907 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:49.804912 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:49.805180 (+ 268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:49.807173 (+ 1993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:49.807174 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:49.807174 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:49.807399 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:49.807599 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:49.807600 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:49.807601 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:49.807668 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:49.807962 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:49.807964 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:49.808018 (+ 54us) client_negotiation.cc:769] Sending connection context
0928 14:42:49.808093 (+ 75us) client_negotiation.cc:240] Negotiation successful
0928 14:42:49.808221 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":106,"threads_started":1}
Sep 28 14:42:49 dist-test-slave-pv0w krb5kdc[11035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 14:42:49 dist-test-slave-pv0w krb5kdc[11035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:42:49.830441 11130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:49.823107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46824 (local address 127.9.33.126:46127)
0928 14:42:49.823159 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:49.823161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:49.823170 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:42:49.823225 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:49.823226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:49.823236 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:49.823285 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:49.823939 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.824456 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.825173 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.825347 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.825412 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:49.827585 (+ 2173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:49.827586 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:49.827587 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:49.827594 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:49.829314 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:49.829682 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:49.829683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:49.829684 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:49.829742 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:49.829932 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:49.829933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:49.829933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:49.830053 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:49.830138 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:49.830224 (+ 86us) server_negotiation.cc:299] Negotiation successful
0928 14:42:49.830298 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260928 14:42:49.832286 11079 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46824:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:42:49.834373 11079 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:42:49.841214 11585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:49.837363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40144 (local address 127.9.33.65:46725)
0928 14:42:49.837559 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:49.837565 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:49.837579 (+ 14us) server_negotiation.cc:407] Connection header received
0928 14:42:49.837628 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:49.837630 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:49.837650 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:49.837739 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:42:49.837908 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.838926 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.840178 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.840676 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.840815 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:49.840896 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:49.841075 (+ 179us) server_negotiation.cc:299] Negotiation successful
0928 14:42:49.841113 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":84,"threads_started":1}
I20260928 14:42:49.841238 11581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:49.837080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:46725 (local address 127.0.0.1:40144)
0928 14:42:49.837260 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:49.837307 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:49.837400 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:49.837752 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:49.837755 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:49.837758 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:42:49.837838 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:49.837840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.839007 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.839008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:49.840054 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:49.840057 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.840818 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.840819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:49.840952 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:49.840964 (+ 12us) client_negotiation.cc:769] Sending connection context
0928 14:42:49.841053 (+ 89us) client_negotiation.cc:240] Negotiation successful
0928 14:42:49.841112 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":103,"threads_started":1}
I20260928 14:42:49.841945 11208 tablet_service.cc:1467] Processing CreateTablet for tablet 81fa674baf7e4906bcd869cf71ee5ad3 (DEFAULT_TABLE table=test-table [id=562f104592484bc9b04cf01baf1a3add]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:42:49.842353 11208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81fa674baf7e4906bcd869cf71ee5ad3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:49.842489 11582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:49.837082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:36645 (local address 127.0.0.1:37502)
0928 14:42:49.837510 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:49.837518 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:49.837554 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:49.838487 (+ 933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:49.838488 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:49.838490 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:42:49.838538 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:49.838540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.839902 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.839903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:49.841096 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:49.841099 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.842198 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.842199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:49.842269 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:49.842273 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:42:49.842341 (+ 68us) client_negotiation.cc:240] Negotiation successful
0928 14:42:49.842391 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":57,"threads_started":1}
I20260928 14:42:49.842770 11580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:49.837121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:39601 (local address 127.0.0.1:47546)
0928 14:42:49.837582 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:49.837594 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:49.837653 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:49.838173 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:49.838174 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:49.838177 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:42:49.838237 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:49.838239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.839689 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.839690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:49.841097 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:49.841099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.842526 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.842527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:49.842600 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:49.842605 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:42:49.842663 (+ 58us) client_negotiation.cc:240] Negotiation successful
0928 14:42:49.842706 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"mutex_wait_us":4,"thread_start_us":88,"threads_started":1}
I20260928 14:42:49.842841 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:49.837352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37502 (local address 127.9.33.66:36645)
0928 14:42:49.838239 (+ 887us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:49.838244 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:49.838255 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:42:49.838310 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:49.838312 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:49.838329 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:49.838412 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:42:49.838627 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.839814 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.841202 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.842029 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.842103 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:49.842159 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:49.842684 (+ 525us) server_negotiation.cc:299] Negotiation successful
0928 14:42:49.842735 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":89,"threads_started":1}
I20260928 14:42:49.843128 11583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:49.837347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47546 (local address 127.9.33.67:39601)
0928 14:42:49.838007 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:49.838012 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:49.838025 (+ 13us) server_negotiation.cc:407] Connection header received
0928 14:42:49.838062 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:49.838064 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:49.838077 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:49.838175 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:42:49.838328 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.839589 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.841202 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:49.842371 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:49.842435 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:49.842490 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:49.842978 (+ 488us) server_negotiation.cc:299] Negotiation successful
0928 14:42:49.843022 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":85,"threads_started":1}
I20260928 14:42:49.843499 11357 tablet_service.cc:1467] Processing CreateTablet for tablet 81fa674baf7e4906bcd869cf71ee5ad3 (DEFAULT_TABLE table=test-table [id=562f104592484bc9b04cf01baf1a3add]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:42:49.843693 11497 tablet_service.cc:1467] Processing CreateTablet for tablet 81fa674baf7e4906bcd869cf71ee5ad3 (DEFAULT_TABLE table=test-table [id=562f104592484bc9b04cf01baf1a3add]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:42:49.843906 11357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81fa674baf7e4906bcd869cf71ee5ad3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:49.844122 11497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81fa674baf7e4906bcd869cf71ee5ad3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:49.846312 11586 tablet_bootstrap.cc:492] T 81fa674baf7e4906bcd869cf71ee5ad3 P 6475771aee444d78832d41ed481d4255: Bootstrap starting.
I20260928 14:42:49.847064 11587 tablet_bootstrap.cc:492] T 81fa674baf7e4906bcd869cf71ee5ad3 P 4cfa07b117ef40fca06be7d703b98707: Bootstrap starting.
I20260928 14:42:49.847092 11586 tablet_bootstrap.cc:654] T 81fa674baf7e4906bcd869cf71ee5ad3 P 6475771aee444d78832d41ed481d4255: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:49.847497 11586 log.cc:824] T 81fa674baf7e4906bcd869cf71ee5ad3 P 6475771aee444d78832d41ed481d4255: Log is configured to *not* fsync() on all Append() calls
I20260928 14:42:49.847679 11588 tablet_bootstrap.cc:492] T 81fa674baf7e4906bcd869cf71ee5ad3 P 8f0bbb4eaedd49b9b5a0d487b3860300: Bootstrap starting.
I20260928 14:42:49.848052 11587 tablet_bootstrap.cc:654] T 81fa674baf7e4906bcd869cf71ee5ad3 P 4cfa07b117ef40fca06be7d703b98707: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:49.848363 11587 log.cc:824] T 81fa674baf7e4906bcd869cf71ee5ad3 P 4cfa07b117ef40fca06be7d703b98707: Log is configured to *not* fsync() on all Append() calls
I20260928 14:42:49.848388 11588 tablet_bootstrap.cc:654] T 81fa674baf7e4906bcd869cf71ee5ad3 P 8f0bbb4eaedd49b9b5a0d487b3860300: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:49.848773 11588 log.cc:824] T 81fa674baf7e4906bcd869cf71ee5ad3 P 8f0bbb4eaedd49b9b5a0d487b3860300: Log is configured to *not* fsync() on all Append() calls
I20260928 14:42:49.849135 11587 tablet_bootstrap.cc:492] T 81fa674baf7e4906bcd869cf71ee5ad3 P 4cfa07b117ef40fca06be7d703b98707: No bootstrap required, opened a new log
I20260928 14:42:49.849189 11587 ts_tablet_manager.cc:1397] T 81fa674baf7e4906bcd869cf71ee5ad3 P 4cfa07b117ef40fca06be7d703b98707: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:42:49.849592 11586 tablet_bootstrap.cc:492] T 81fa674baf7e4906bcd869cf71ee5ad3 P 6475771aee444d78832d41ed481d4255: No bootstrap required, opened a new log
I20260928 14:42:49.849710 11586 ts_tablet_manager.cc:1397] T 81fa674baf7e4906bcd869cf71ee5ad3 P 6475771aee444d78832d41ed481d4255: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260928 14:42:49.850677 11587 raft_consensus.cc:348] T 81fa674baf7e4906bcd869cf71ee5ad3 P 4cfa07b117ef40fca06be7d703b98707 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } }
I20260928 14:42:49.850812 11587 raft_consensus.cc:374] T 81fa674baf7e4906bcd869cf71ee5ad3 P 4cfa07b117ef40fca06be7d703b98707 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:49.850834 11587 raft_consensus.cc:729] T 81fa674baf7e4906bcd869cf71ee5ad3 P 4cfa07b117ef40fca06be7d703b98707 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cfa07b117ef40fca06be7d703b98707, State: Initialized, Role: FOLLOWER
I20260928 14:42:49.850963 11587 consensus_queue.cc:260] T 81fa674baf7e4906bcd869cf71ee5ad3 P 4cfa07b117ef40fca06be7d703b98707 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } }
I20260928 14:42:49.851239 11587 ts_tablet_manager.cc:1428] T 81fa674baf7e4906bcd869cf71ee5ad3 P 4cfa07b117ef40fca06be7d703b98707: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:42:49.851830 11563 heartbeater.cc:503] Master 127.9.33.126:46127 was elected leader, sending a full tablet report...
I20260928 14:42:49.851689 11586 raft_consensus.cc:348] T 81fa674baf7e4906bcd869cf71ee5ad3 P 6475771aee444d78832d41ed481d4255 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } }
I20260928 14:42:49.851908 11586 raft_consensus.cc:374] T 81fa674baf7e4906bcd869cf71ee5ad3 P 6475771aee444d78832d41ed481d4255 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:49.851956 11586 raft_consensus.cc:729] T 81fa674baf7e4906bcd869cf71ee5ad3 P 6475771aee444d78832d41ed481d4255 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6475771aee444d78832d41ed481d4255, State: Initialized, Role: FOLLOWER
I20260928 14:42:49.852080 11586 consensus_queue.cc:260] T 81fa674baf7e4906bcd869cf71ee5ad3 P 6475771aee444d78832d41ed481d4255 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } }
I20260928 14:42:49.852468 11283 heartbeater.cc:503] Master 127.9.33.126:46127 was elected leader, sending a full tablet report...
I20260928 14:42:49.852584 11588 tablet_bootstrap.cc:492] T 81fa674baf7e4906bcd869cf71ee5ad3 P 8f0bbb4eaedd49b9b5a0d487b3860300: No bootstrap required, opened a new log
I20260928 14:42:49.852654 11588 ts_tablet_manager.cc:1397] T 81fa674baf7e4906bcd869cf71ee5ad3 P 8f0bbb4eaedd49b9b5a0d487b3860300: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260928 14:42:49.852752 11586 ts_tablet_manager.cc:1428] T 81fa674baf7e4906bcd869cf71ee5ad3 P 6475771aee444d78832d41ed481d4255: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:42:49.854322 11588 raft_consensus.cc:348] T 81fa674baf7e4906bcd869cf71ee5ad3 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } }
I20260928 14:42:49.854439 11588 raft_consensus.cc:374] T 81fa674baf7e4906bcd869cf71ee5ad3 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:49.854460 11588 raft_consensus.cc:729] T 81fa674baf7e4906bcd869cf71ee5ad3 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f0bbb4eaedd49b9b5a0d487b3860300, State: Initialized, Role: FOLLOWER
I20260928 14:42:49.854542 11588 consensus_queue.cc:260] T 81fa674baf7e4906bcd869cf71ee5ad3 P 8f0bbb4eaedd49b9b5a0d487b3860300 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } }
I20260928 14:42:49.854745 11588 ts_tablet_manager.cc:1428] T 81fa674baf7e4906bcd869cf71ee5ad3 P 8f0bbb4eaedd49b9b5a0d487b3860300: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:42:49.854821 11423 heartbeater.cc:503] Master 127.9.33.126:46127 was elected leader, sending a full tablet report...
W20260928 14:42:49.882604 11424 tablet.cc:2369] T 81fa674baf7e4906bcd869cf71ee5ad3 P 8f0bbb4eaedd49b9b5a0d487b3860300: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:42:49.966449 11284 tablet.cc:2369] T 81fa674baf7e4906bcd869cf71ee5ad3 P 6475771aee444d78832d41ed481d4255: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:42:50.048950 11564 tablet.cc:2369] T 81fa674baf7e4906bcd869cf71ee5ad3 P 4cfa07b117ef40fca06be7d703b98707: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:42:50.068501 11594 raft_consensus.cc:482] T 81fa674baf7e4906bcd869cf71ee5ad3 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:42:50.068645 11594 raft_consensus.cc:504] T 81fa674baf7e4906bcd869cf71ee5ad3 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } }
I20260928 14:42:50.069010 11594 leader_election.cc:288] T 81fa674baf7e4906bcd869cf71ee5ad3 P 8f0bbb4eaedd49b9b5a0d487b3860300 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6475771aee444d78832d41ed481d4255 (127.9.33.65:46725), 4cfa07b117ef40fca06be7d703b98707 (127.9.33.67:39601)
I20260928 14:42:50.072029 11585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:50.069292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:39207 (local address 127.9.33.65:46725)
0928 14:42:50.069355 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:50.069357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:50.069393 (+ 36us) server_negotiation.cc:407] Connection header received
0928 14:42:50.069424 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:50.069425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:50.069437 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:50.069495 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:42:50.069694 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:50.070249 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:50.071267 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:50.071641 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:50.071727 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:50.071758 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:50.071910 (+ 152us) server_negotiation.cc:299] Negotiation successful
0928 14:42:50.071955 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260928 14:42:50.072031 11316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:50.069223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:46725 (local address 127.9.33.66:39207)
0928 14:42:50.069293 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:50.069305 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:50.069350 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:50.069532 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:50.069533 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:50.069536 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:42:50.069614 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:50.069616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:50.070329 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:50.070330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:50.071160 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:50.071163 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:50.071747 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:50.071748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:50.071824 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:50.071828 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:42:50.071881 (+ 53us) client_negotiation.cc:240] Negotiation successful
0928 14:42:50.071942 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260928 14:42:50.072065 11583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:50.069301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:35843 (local address 127.9.33.67:39601)
0928 14:42:50.069347 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:50.069349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:50.069520 (+ 171us) server_negotiation.cc:407] Connection header received
0928 14:42:50.069549 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:50.069549 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:50.069561 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:50.069624 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:42:50.069848 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:50.070391 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:50.071307 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:50.071646 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:50.071766 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:50.071777 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:50.071955 (+ 178us) server_negotiation.cc:299] Negotiation successful
0928 14:42:50.071999 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 14:42:50.072049 11595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:50.069229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:39601 (local address 127.9.33.66:35843)
0928 14:42:50.069431 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:50.069442 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:50.069502 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:50.069632 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:50.069633 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:50.069634 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:42:50.069749 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:50.069751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:50.070472 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:50.070473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:50.071179 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:50.071181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:50.071775 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:50.071776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:50.071850 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:50.071853 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:42:50.071930 (+ 77us) client_negotiation.cc:240] Negotiation successful
0928 14:42:50.071964 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":47,"thread_start_us":103,"threads_started":1}
I20260928 14:42:50.072471 11237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81fa674baf7e4906bcd869cf71ee5ad3" candidate_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6475771aee444d78832d41ed481d4255" is_pre_election: true
I20260928 14:42:50.072669 11237 raft_consensus.cc:2454] T 81fa674baf7e4906bcd869cf71ee5ad3 P 6475771aee444d78832d41ed481d4255 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f0bbb4eaedd49b9b5a0d487b3860300 in term 0.
I20260928 14:42:50.072656 11517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81fa674baf7e4906bcd869cf71ee5ad3" candidate_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cfa07b117ef40fca06be7d703b98707" is_pre_election: true
I20260928 14:42:50.072785 11517 raft_consensus.cc:2454] T 81fa674baf7e4906bcd869cf71ee5ad3 P 4cfa07b117ef40fca06be7d703b98707 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f0bbb4eaedd49b9b5a0d487b3860300 in term 0.
I20260928 14:42:50.072932 11311 leader_election.cc:302] T 81fa674baf7e4906bcd869cf71ee5ad3 P 8f0bbb4eaedd49b9b5a0d487b3860300 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6475771aee444d78832d41ed481d4255, 8f0bbb4eaedd49b9b5a0d487b3860300; no voters:
I20260928 14:42:50.073212 11594 raft_consensus.cc:2781] T 81fa674baf7e4906bcd869cf71ee5ad3 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:42:50.073289 11594 raft_consensus.cc:482] T 81fa674baf7e4906bcd869cf71ee5ad3 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:42:50.073323 11594 raft_consensus.cc:3037] T 81fa674baf7e4906bcd869cf71ee5ad3 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:50.074294 11594 raft_consensus.cc:504] T 81fa674baf7e4906bcd869cf71ee5ad3 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } }
I20260928 14:42:50.074453 11594 leader_election.cc:288] T 81fa674baf7e4906bcd869cf71ee5ad3 P 8f0bbb4eaedd49b9b5a0d487b3860300 [CANDIDATE]: Term 1 election: Requested vote from peers 6475771aee444d78832d41ed481d4255 (127.9.33.65:46725), 4cfa07b117ef40fca06be7d703b98707 (127.9.33.67:39601)
I20260928 14:42:50.074606 11237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81fa674baf7e4906bcd869cf71ee5ad3" candidate_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6475771aee444d78832d41ed481d4255"
I20260928 14:42:50.074607 11517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81fa674baf7e4906bcd869cf71ee5ad3" candidate_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cfa07b117ef40fca06be7d703b98707"
I20260928 14:42:50.074707 11517 raft_consensus.cc:3037] T 81fa674baf7e4906bcd869cf71ee5ad3 P 4cfa07b117ef40fca06be7d703b98707 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:50.074712 11237 raft_consensus.cc:3037] T 81fa674baf7e4906bcd869cf71ee5ad3 P 6475771aee444d78832d41ed481d4255 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:50.075572 11517 raft_consensus.cc:2454] T 81fa674baf7e4906bcd869cf71ee5ad3 P 4cfa07b117ef40fca06be7d703b98707 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f0bbb4eaedd49b9b5a0d487b3860300 in term 1.
I20260928 14:42:50.075572 11237 raft_consensus.cc:2454] T 81fa674baf7e4906bcd869cf71ee5ad3 P 6475771aee444d78832d41ed481d4255 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f0bbb4eaedd49b9b5a0d487b3860300 in term 1.
I20260928 14:42:50.075779 11308 leader_election.cc:302] T 81fa674baf7e4906bcd869cf71ee5ad3 P 8f0bbb4eaedd49b9b5a0d487b3860300 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cfa07b117ef40fca06be7d703b98707, 8f0bbb4eaedd49b9b5a0d487b3860300; no voters:
I20260928 14:42:50.075875 11594 raft_consensus.cc:2781] T 81fa674baf7e4906bcd869cf71ee5ad3 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:42:50.076025 11594 raft_consensus.cc:686] T 81fa674baf7e4906bcd869cf71ee5ad3 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 1 LEADER]: Becoming Leader. State: Replica: 8f0bbb4eaedd49b9b5a0d487b3860300, State: Running, Role: LEADER
I20260928 14:42:50.076135 11594 consensus_queue.cc:237] T 81fa674baf7e4906bcd869cf71ee5ad3 P 8f0bbb4eaedd49b9b5a0d487b3860300 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } }
I20260928 14:42:50.077011 11079 catalog_manager.cc:5495] T 81fa674baf7e4906bcd869cf71ee5ad3 P 8f0bbb4eaedd49b9b5a0d487b3860300 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f0bbb4eaedd49b9b5a0d487b3860300 (127.9.33.66). New cstate: current_term: 1 leader_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } health_report { overall_health: HEALTHY } } }
Sep 28 14:42:50 dist-test-slave-pv0w krb5kdc[11035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606569, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:42:50.156316 11603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:50.148534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46127 (local address 127.0.0.1:46832)
0928 14:42:50.148708 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:50.148719 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:50.148837 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:50.149005 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:50.149006 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:50.149373 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:50.149427 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:50.149429 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:50.150195 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:50.150196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:50.150848 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:50.150850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:50.151277 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:50.151278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:50.151341 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:50.151969 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:50.151974 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:50.153525 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:50.155117 (+ 1592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:50.155125 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:50.155128 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:50.155343 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:50.155582 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:50.155582 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:50.155582 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:50.155640 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:50.156005 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:50.156011 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:50.156059 (+ 48us) client_negotiation.cc:769] Sending connection context
0928 14:42:50.156118 (+ 59us) client_negotiation.cc:240] Negotiation successful
0928 14:42:50.156217 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":63,"threads_started":1}
I20260928 14:42:50.156394 11130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:50.148613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46832 (local address 127.9.33.126:46127)
0928 14:42:50.148672 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:50.148674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:50.148821 (+ 147us) server_negotiation.cc:407] Connection header received
0928 14:42:50.148899 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:50.148900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:50.148918 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:50.148999 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:50.149526 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:50.150098 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:50.150945 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:50.151159 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:50.151266 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:50.153618 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:50.153620 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:50.153620 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:50.153628 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:50.155029 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:50.155435 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:50.155436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:50.155437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:50.155512 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:50.155783 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:50.155788 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:50.155789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:50.155936 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:50.156000 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:50.156155 (+ 155us) server_negotiation.cc:299] Negotiation successful
0928 14:42:50.156267 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":28}
I20260928 14:42:50.158459 11079 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:46832:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260928 14:42:50.158700 11079 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:42:50.160415 11208 tablet_service.cc:1467] Processing CreateTablet for tablet a9046029b7f24a9a85a7fd31e550effe (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fb32c3abf928423caab9810be5b233ac]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 14:42:50.160445 11357 tablet_service.cc:1467] Processing CreateTablet for tablet a9046029b7f24a9a85a7fd31e550effe (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fb32c3abf928423caab9810be5b233ac]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 14:42:50.160516 11497 tablet_service.cc:1467] Processing CreateTablet for tablet a9046029b7f24a9a85a7fd31e550effe (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fb32c3abf928423caab9810be5b233ac]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 14:42:50.160614 11208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9046029b7f24a9a85a7fd31e550effe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:50.160614 11357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9046029b7f24a9a85a7fd31e550effe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:50.160660 11497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9046029b7f24a9a85a7fd31e550effe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:50.161864 11586 tablet_bootstrap.cc:492] T a9046029b7f24a9a85a7fd31e550effe P 6475771aee444d78832d41ed481d4255: Bootstrap starting.
I20260928 14:42:50.162168 11587 tablet_bootstrap.cc:492] T a9046029b7f24a9a85a7fd31e550effe P 4cfa07b117ef40fca06be7d703b98707: Bootstrap starting.
I20260928 14:42:50.162173 11588 tablet_bootstrap.cc:492] T a9046029b7f24a9a85a7fd31e550effe P 8f0bbb4eaedd49b9b5a0d487b3860300: Bootstrap starting.
I20260928 14:42:50.162282 11586 tablet_bootstrap.cc:654] T a9046029b7f24a9a85a7fd31e550effe P 6475771aee444d78832d41ed481d4255: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:50.162561 11588 tablet_bootstrap.cc:654] T a9046029b7f24a9a85a7fd31e550effe P 8f0bbb4eaedd49b9b5a0d487b3860300: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:50.162685 11587 tablet_bootstrap.cc:654] T a9046029b7f24a9a85a7fd31e550effe P 4cfa07b117ef40fca06be7d703b98707: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:50.162962 11586 tablet_bootstrap.cc:492] T a9046029b7f24a9a85a7fd31e550effe P 6475771aee444d78832d41ed481d4255: No bootstrap required, opened a new log
I20260928 14:42:50.163029 11586 ts_tablet_manager.cc:1397] T a9046029b7f24a9a85a7fd31e550effe P 6475771aee444d78832d41ed481d4255: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 14:42:50.163218 11586 raft_consensus.cc:348] T a9046029b7f24a9a85a7fd31e550effe P 6475771aee444d78832d41ed481d4255 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } }
I20260928 14:42:50.163272 11588 tablet_bootstrap.cc:492] T a9046029b7f24a9a85a7fd31e550effe P 8f0bbb4eaedd49b9b5a0d487b3860300: No bootstrap required, opened a new log
I20260928 14:42:50.163298 11586 raft_consensus.cc:374] T a9046029b7f24a9a85a7fd31e550effe P 6475771aee444d78832d41ed481d4255 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:50.163321 11586 raft_consensus.cc:729] T a9046029b7f24a9a85a7fd31e550effe P 6475771aee444d78832d41ed481d4255 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6475771aee444d78832d41ed481d4255, State: Initialized, Role: FOLLOWER
I20260928 14:42:50.163324 11588 ts_tablet_manager.cc:1397] T a9046029b7f24a9a85a7fd31e550effe P 8f0bbb4eaedd49b9b5a0d487b3860300: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 14:42:50.163372 11586 consensus_queue.cc:260] T a9046029b7f24a9a85a7fd31e550effe P 6475771aee444d78832d41ed481d4255 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } }
I20260928 14:42:50.163486 11586 ts_tablet_manager.cc:1428] T a9046029b7f24a9a85a7fd31e550effe P 6475771aee444d78832d41ed481d4255: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260928 14:42:50.163463 11588 raft_consensus.cc:348] T a9046029b7f24a9a85a7fd31e550effe P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } }
I20260928 14:42:50.163527 11588 raft_consensus.cc:374] T a9046029b7f24a9a85a7fd31e550effe P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:50.163561 11588 raft_consensus.cc:729] T a9046029b7f24a9a85a7fd31e550effe P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f0bbb4eaedd49b9b5a0d487b3860300, State: Initialized, Role: FOLLOWER
I20260928 14:42:50.163610 11588 consensus_queue.cc:260] T a9046029b7f24a9a85a7fd31e550effe P 8f0bbb4eaedd49b9b5a0d487b3860300 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } }
I20260928 14:42:50.163623 11593 tablet_replica.cc:439] T a9046029b7f24a9a85a7fd31e550effe P 6475771aee444d78832d41ed481d4255: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } } }
I20260928 14:42:50.163702 11593 tablet_replica.cc:442] T a9046029b7f24a9a85a7fd31e550effe P 6475771aee444d78832d41ed481d4255: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:42:50.163745 11588 ts_tablet_manager.cc:1428] T a9046029b7f24a9a85a7fd31e550effe P 8f0bbb4eaedd49b9b5a0d487b3860300: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260928 14:42:50.163997 11594 tablet_replica.cc:439] T a9046029b7f24a9a85a7fd31e550effe P 8f0bbb4eaedd49b9b5a0d487b3860300: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } } }
I20260928 14:42:50.164099 11594 tablet_replica.cc:442] T a9046029b7f24a9a85a7fd31e550effe P 8f0bbb4eaedd49b9b5a0d487b3860300: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:42:50.164603 11587 tablet_bootstrap.cc:492] T a9046029b7f24a9a85a7fd31e550effe P 4cfa07b117ef40fca06be7d703b98707: No bootstrap required, opened a new log
I20260928 14:42:50.164649 11587 ts_tablet_manager.cc:1397] T a9046029b7f24a9a85a7fd31e550effe P 4cfa07b117ef40fca06be7d703b98707: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:42:50.164786 11587 raft_consensus.cc:348] T a9046029b7f24a9a85a7fd31e550effe P 4cfa07b117ef40fca06be7d703b98707 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } }
I20260928 14:42:50.164868 11587 raft_consensus.cc:374] T a9046029b7f24a9a85a7fd31e550effe P 4cfa07b117ef40fca06be7d703b98707 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:50.164883 11587 raft_consensus.cc:729] T a9046029b7f24a9a85a7fd31e550effe P 4cfa07b117ef40fca06be7d703b98707 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cfa07b117ef40fca06be7d703b98707, State: Initialized, Role: FOLLOWER
I20260928 14:42:50.164935 11587 consensus_queue.cc:260] T a9046029b7f24a9a85a7fd31e550effe P 4cfa07b117ef40fca06be7d703b98707 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } }
I20260928 14:42:50.165066 11587 ts_tablet_manager.cc:1428] T a9046029b7f24a9a85a7fd31e550effe P 4cfa07b117ef40fca06be7d703b98707: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:50.165160 11592 tablet_replica.cc:439] T a9046029b7f24a9a85a7fd31e550effe P 4cfa07b117ef40fca06be7d703b98707: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } } }
I20260928 14:42:50.165304 11592 tablet_replica.cc:442] T a9046029b7f24a9a85a7fd31e550effe P 4cfa07b117ef40fca06be7d703b98707: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:42:50.232479 11592 raft_consensus.cc:482] T a9046029b7f24a9a85a7fd31e550effe P 4cfa07b117ef40fca06be7d703b98707 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:42:50.232625 11592 raft_consensus.cc:504] T a9046029b7f24a9a85a7fd31e550effe P 4cfa07b117ef40fca06be7d703b98707 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } }
I20260928 14:42:50.233050 11592 leader_election.cc:288] T a9046029b7f24a9a85a7fd31e550effe P 4cfa07b117ef40fca06be7d703b98707 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6475771aee444d78832d41ed481d4255 (127.9.33.65:46725), 8f0bbb4eaedd49b9b5a0d487b3860300 (127.9.33.66:36645)
I20260928 14:42:50.236826 11456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:50.233290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:36645 (local address 127.9.33.67:46939)
0928 14:42:50.233366 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:50.233379 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:50.233426 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:50.233603 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:50.233604 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:50.233605 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:42:50.233688 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:50.233690 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:50.234364 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:50.234366 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:50.235637 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:50.235639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:50.236538 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:50.236539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:50.236614 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:50.236618 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:42:50.236700 (+ 82us) client_negotiation.cc:240] Negotiation successful
0928 14:42:50.236746 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260928 14:42:50.236852 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:50.233414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:46939 (local address 127.9.33.66:36645)
0928 14:42:50.233469 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:50.233471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:50.233480 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:42:50.233518 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:50.233519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:50.233528 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:50.233599 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:42:50.233769 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:50.234270 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:50.235792 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:50.236449 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:50.236527 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:50.236538 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:50.236707 (+ 169us) server_negotiation.cc:299] Negotiation successful
0928 14:42:50.236747 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260928 14:42:50.236976 11585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:50.233416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:50047 (local address 127.9.33.65:46725)
0928 14:42:50.233465 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:50.233467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:50.233615 (+ 148us) server_negotiation.cc:407] Connection header received
0928 14:42:50.233786 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:50.233787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:50.233798 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:50.233888 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:42:50.234052 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:50.234544 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:50.235972 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:50.236600 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:50.236678 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:50.236689 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:50.236866 (+ 177us) server_negotiation.cc:299] Negotiation successful
0928 14:42:50.236906 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 14:42:50.236986 11607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:50.233300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:46725 (local address 127.9.33.67:50047)
0928 14:42:50.233522 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:50.233531 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:50.233633 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:50.233893 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:50.233894 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:50.233896 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:42:50.233969 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:50.233971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:50.234672 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:50.234673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:50.235883 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:50.235885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:50.236712 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:50.236713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:50.236791 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:50.236794 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:42:50.236860 (+ 66us) client_negotiation.cc:240] Negotiation successful
0928 14:42:50.236906 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":115,"threads_started":1}
I20260928 14:42:50.237244 11377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9046029b7f24a9a85a7fd31e550effe" candidate_uuid: "4cfa07b117ef40fca06be7d703b98707" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" is_pre_election: true
I20260928 14:42:50.237376 11377 raft_consensus.cc:2454] T a9046029b7f24a9a85a7fd31e550effe P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cfa07b117ef40fca06be7d703b98707 in term 0.
I20260928 14:42:50.237385 11237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9046029b7f24a9a85a7fd31e550effe" candidate_uuid: "4cfa07b117ef40fca06be7d703b98707" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6475771aee444d78832d41ed481d4255" is_pre_election: true
I20260928 14:42:50.237478 11237 raft_consensus.cc:2454] T a9046029b7f24a9a85a7fd31e550effe P 6475771aee444d78832d41ed481d4255 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cfa07b117ef40fca06be7d703b98707 in term 0.
I20260928 14:42:50.237588 11450 leader_election.cc:302] T a9046029b7f24a9a85a7fd31e550effe P 4cfa07b117ef40fca06be7d703b98707 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cfa07b117ef40fca06be7d703b98707, 8f0bbb4eaedd49b9b5a0d487b3860300; no voters:
I20260928 14:42:50.237725 11592 raft_consensus.cc:2781] T a9046029b7f24a9a85a7fd31e550effe P 4cfa07b117ef40fca06be7d703b98707 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:42:50.237773 11592 raft_consensus.cc:482] T a9046029b7f24a9a85a7fd31e550effe P 4cfa07b117ef40fca06be7d703b98707 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:42:50.237797 11592 raft_consensus.cc:3037] T a9046029b7f24a9a85a7fd31e550effe P 4cfa07b117ef40fca06be7d703b98707 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:50.238502 11592 raft_consensus.cc:504] T a9046029b7f24a9a85a7fd31e550effe P 4cfa07b117ef40fca06be7d703b98707 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } }
I20260928 14:42:50.238652 11592 leader_election.cc:288] T a9046029b7f24a9a85a7fd31e550effe P 4cfa07b117ef40fca06be7d703b98707 [CANDIDATE]: Term 1 election: Requested vote from peers 6475771aee444d78832d41ed481d4255 (127.9.33.65:46725), 8f0bbb4eaedd49b9b5a0d487b3860300 (127.9.33.66:36645)
I20260928 14:42:50.238790 11237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9046029b7f24a9a85a7fd31e550effe" candidate_uuid: "4cfa07b117ef40fca06be7d703b98707" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6475771aee444d78832d41ed481d4255"
I20260928 14:42:50.238878 11237 raft_consensus.cc:3037] T a9046029b7f24a9a85a7fd31e550effe P 6475771aee444d78832d41ed481d4255 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:50.238848 11377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9046029b7f24a9a85a7fd31e550effe" candidate_uuid: "4cfa07b117ef40fca06be7d703b98707" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300"
I20260928 14:42:50.238927 11377 raft_consensus.cc:3037] T a9046029b7f24a9a85a7fd31e550effe P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:50.239591 11377 raft_consensus.cc:2454] T a9046029b7f24a9a85a7fd31e550effe P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cfa07b117ef40fca06be7d703b98707 in term 1.
I20260928 14:42:50.239763 11237 raft_consensus.cc:2454] T a9046029b7f24a9a85a7fd31e550effe P 6475771aee444d78832d41ed481d4255 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cfa07b117ef40fca06be7d703b98707 in term 1.
I20260928 14:42:50.239763 11450 leader_election.cc:302] T a9046029b7f24a9a85a7fd31e550effe P 4cfa07b117ef40fca06be7d703b98707 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cfa07b117ef40fca06be7d703b98707, 8f0bbb4eaedd49b9b5a0d487b3860300; no voters:
I20260928 14:42:50.239919 11592 raft_consensus.cc:2781] T a9046029b7f24a9a85a7fd31e550effe P 4cfa07b117ef40fca06be7d703b98707 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:42:50.240108 11592 raft_consensus.cc:686] T a9046029b7f24a9a85a7fd31e550effe P 4cfa07b117ef40fca06be7d703b98707 [term 1 LEADER]: Becoming Leader. State: Replica: 4cfa07b117ef40fca06be7d703b98707, State: Running, Role: LEADER
I20260928 14:42:50.240219 11592 consensus_queue.cc:237] T a9046029b7f24a9a85a7fd31e550effe P 4cfa07b117ef40fca06be7d703b98707 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } }
I20260928 14:42:50.240728 11608 tablet_replica.cc:439] T a9046029b7f24a9a85a7fd31e550effe P 4cfa07b117ef40fca06be7d703b98707: TxnStatusTablet state changed. Reason: New leader 4cfa07b117ef40fca06be7d703b98707. Latest consensus state: current_term: 1 leader_uuid: "4cfa07b117ef40fca06be7d703b98707" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } } }
I20260928 14:42:50.240864 11608 tablet_replica.cc:442] T a9046029b7f24a9a85a7fd31e550effe P 4cfa07b117ef40fca06be7d703b98707: This TxnStatusTablet replica's current role is: LEADER
I20260928 14:42:50.241070 11610 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260928 14:42:50.241101 11079 catalog_manager.cc:5495] T a9046029b7f24a9a85a7fd31e550effe P 4cfa07b117ef40fca06be7d703b98707 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4cfa07b117ef40fca06be7d703b98707 (127.9.33.67). New cstate: current_term: 1 leader_uuid: "4cfa07b117ef40fca06be7d703b98707" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } health_report { overall_health: HEALTHY } } }
I20260928 14:42:50.241192 11610 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260928 14:42:50.260623 11603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:50.257096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:39601 (local address 127.0.0.1:47552)
0928 14:42:50.257182 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:50.257196 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:50.257253 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:50.257442 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:50.257442 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:50.257443 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 14:42:50.257490 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:50.257493 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:50.258931 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:50.258932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:50.259516 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:50.259518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:50.260035 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:50.260036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:50.260096 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:50.260113 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 14:42:50.260446 (+ 333us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 14:42:50.260457 (+ 11us) client_negotiation.cc:769] Sending connection context
0928 14:42:50.260511 (+ 54us) client_negotiation.cc:240] Negotiation successful
0928 14:42:50.260565 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260928 14:42:50.260637 11583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:50.257209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47552 (local address 127.9.33.67:39601)
0928 14:42:50.257282 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:50.257284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:50.257309 (+ 25us) server_negotiation.cc:407] Connection header received
0928 14:42:50.257342 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:50.257343 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:50.257364 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:50.257429 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:42:50.257556 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:50.258818 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:50.259607 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:50.259951 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:50.260023 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:50.260215 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:42:50.260345 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:42:50.260429 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:50.260516 (+ 87us) server_negotiation.cc:299] Negotiation successful
0928 14:42:50.260539 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260928 14:42:50.262563 11377 raft_consensus.cc:1261] T a9046029b7f24a9a85a7fd31e550effe P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 1 FOLLOWER]: Refusing update from remote peer 4cfa07b117ef40fca06be7d703b98707: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:42:50.262547 11237 raft_consensus.cc:1261] T a9046029b7f24a9a85a7fd31e550effe P 6475771aee444d78832d41ed481d4255 [term 1 FOLLOWER]: Refusing update from remote peer 4cfa07b117ef40fca06be7d703b98707: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:42:50.262887 11592 consensus_queue.cc:1035] T a9046029b7f24a9a85a7fd31e550effe P 4cfa07b117ef40fca06be7d703b98707 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:42:50.263036 11608 consensus_queue.cc:1035] T a9046029b7f24a9a85a7fd31e550effe P 4cfa07b117ef40fca06be7d703b98707 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:42:50.264170 11594 tablet_replica.cc:439] T a9046029b7f24a9a85a7fd31e550effe P 8f0bbb4eaedd49b9b5a0d487b3860300: TxnStatusTablet state changed. Reason: New leader 4cfa07b117ef40fca06be7d703b98707. Latest consensus state: current_term: 1 leader_uuid: "4cfa07b117ef40fca06be7d703b98707" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } } }
I20260928 14:42:50.264192 11593 tablet_replica.cc:439] T a9046029b7f24a9a85a7fd31e550effe P 6475771aee444d78832d41ed481d4255: TxnStatusTablet state changed. Reason: New leader 4cfa07b117ef40fca06be7d703b98707. Latest consensus state: current_term: 1 leader_uuid: "4cfa07b117ef40fca06be7d703b98707" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } } }
I20260928 14:42:50.264264 11594 tablet_replica.cc:442] T a9046029b7f24a9a85a7fd31e550effe P 8f0bbb4eaedd49b9b5a0d487b3860300: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:42:50.264288 11593 tablet_replica.cc:442] T a9046029b7f24a9a85a7fd31e550effe P 6475771aee444d78832d41ed481d4255: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:42:50.265257 11608 tablet_replica.cc:439] T a9046029b7f24a9a85a7fd31e550effe P 4cfa07b117ef40fca06be7d703b98707: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4cfa07b117ef40fca06be7d703b98707" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } } }
I20260928 14:42:50.265264 11592 tablet_replica.cc:439] T a9046029b7f24a9a85a7fd31e550effe P 4cfa07b117ef40fca06be7d703b98707: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4cfa07b117ef40fca06be7d703b98707" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } } }
I20260928 14:42:50.265353 11608 tablet_replica.cc:442] T a9046029b7f24a9a85a7fd31e550effe P 4cfa07b117ef40fca06be7d703b98707: This TxnStatusTablet replica's current role is: LEADER
I20260928 14:42:50.265369 11592 tablet_replica.cc:442] T a9046029b7f24a9a85a7fd31e550effe P 4cfa07b117ef40fca06be7d703b98707: This TxnStatusTablet replica's current role is: LEADER
I20260928 14:42:50.265409 11594 tablet_replica.cc:439] T a9046029b7f24a9a85a7fd31e550effe P 8f0bbb4eaedd49b9b5a0d487b3860300: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4cfa07b117ef40fca06be7d703b98707" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } } }
I20260928 14:42:50.265483 11594 tablet_replica.cc:442] T a9046029b7f24a9a85a7fd31e550effe P 8f0bbb4eaedd49b9b5a0d487b3860300: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:42:50.265934 11593 tablet_replica.cc:439] T a9046029b7f24a9a85a7fd31e550effe P 6475771aee444d78832d41ed481d4255: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4cfa07b117ef40fca06be7d703b98707" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } } }
I20260928 14:42:50.266011 11593 tablet_replica.cc:442] T a9046029b7f24a9a85a7fd31e550effe P 6475771aee444d78832d41ed481d4255: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:42:50.266178 11614 mvcc.cc:205] Tried to move back new op lower bound from 7334324511792615424 to 7334324511704559616. Current Snapshot: MvccSnapshot[applied={T|T < 7334324511792615424}]
I20260928 14:42:50.266462 11615 mvcc.cc:205] Tried to move back new op lower bound from 7334324511792615424 to 7334324511704559616. Current Snapshot: MvccSnapshot[applied={T|T < 7334324511792615424}]
I20260928 14:42:50.270947 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:50.268133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37506 (local address 127.9.33.66:36645)
0928 14:42:50.268195 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:50.268197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:50.268251 (+ 54us) server_negotiation.cc:407] Connection header received
0928 14:42:50.268336 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:50.268337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:50.268351 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:50.268405 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:42:50.268523 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:50.269059 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:50.270164 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:50.270362 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:50.270413 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:50.270613 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:42:50.270708 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:42:50.270774 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:50.270842 (+ 68us) server_negotiation.cc:299] Negotiation successful
0928 14:42:50.270862 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 14:42:50.276242 11583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:50.273544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:49401 (local address 127.9.33.67:39601)
0928 14:42:50.273586 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:50.273589 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:50.273729 (+ 140us) server_negotiation.cc:407] Connection header received
0928 14:42:50.273775 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:50.273776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:50.273788 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:50.273854 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:42:50.273950 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:50.274658 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:50.275416 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:50.275713 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:50.275820 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:50.276048 (+ 228us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:42:50.276077 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:42:50.276120 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:50.276173 (+ 53us) server_negotiation.cc:299] Negotiation successful
0928 14:42:50.276189 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 14:42:50.276316 11625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:50.273484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:39601 (local address 127.9.33.66:49401)
0928 14:42:50.273650 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:50.273661 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:50.273722 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:50.273854 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:50.273855 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:50.273856 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 14:42:50.273913 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:50.273914 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:50.274750 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:50.274751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:50.275337 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:50.275339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:50.275858 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:50.275859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:50.275928 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:50.275946 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 14:42:50.276121 (+ 175us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 14:42:50.276122 (+ 1us) client_negotiation.cc:769] Sending connection context
0928 14:42:50.276171 (+ 49us) client_negotiation.cc:240] Negotiation successful
0928 14:42:50.276218 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":104,"threads_started":1}
I20260928 14:42:50.278601 11237 raft_consensus.cc:1261] T 81fa674baf7e4906bcd869cf71ee5ad3 P 6475771aee444d78832d41ed481d4255 [term 1 FOLLOWER]: Refusing update from remote peer 8f0bbb4eaedd49b9b5a0d487b3860300: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:42:50.278673 11517 raft_consensus.cc:1261] T 81fa674baf7e4906bcd869cf71ee5ad3 P 4cfa07b117ef40fca06be7d703b98707 [term 1 FOLLOWER]: Refusing update from remote peer 8f0bbb4eaedd49b9b5a0d487b3860300: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:42:50.278859 11596 consensus_queue.cc:1035] T 81fa674baf7e4906bcd869cf71ee5ad3 P 8f0bbb4eaedd49b9b5a0d487b3860300 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:42:50.279088 11596 consensus_queue.cc:1035] T 81fa674baf7e4906bcd869cf71ee5ad3 P 8f0bbb4eaedd49b9b5a0d487b3860300 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:42:50.285035 11613 mvcc.cc:205] Tried to move back new op lower bound from 7334324511859380224 to 7334324511032479744. Current Snapshot: MvccSnapshot[applied={T|T < 7334324511859380224}]
I20260928 14:42:50.288005 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:50.285395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:47379 (local address 127.9.33.66:36645)
0928 14:42:50.285434 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:50.285436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:50.285476 (+ 40us) server_negotiation.cc:407] Connection header received
0928 14:42:50.285537 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:50.285538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:50.285548 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:50.285608 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:42:50.285783 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:50.286413 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:50.287187 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:50.287476 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:50.287553 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:50.287712 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:42:50.287774 (+ 62us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:42:50.287834 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:50.287918 (+ 84us) server_negotiation.cc:299] Negotiation successful
0928 14:42:50.287938 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 14:42:50.288020 11569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:50.285360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:36645 (local address 127.9.33.67:47379)
0928 14:42:50.285431 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:50.285442 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:50.285490 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:50.285625 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:50.285626 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:50.285627 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 14:42:50.285701 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:50.285702 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:50.286498 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:50.286499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:50.287100 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:50.287102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:50.287552 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:50.287553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:50.287624 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:50.287640 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 14:42:50.287833 (+ 193us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 14:42:50.287835 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 14:42:50.287911 (+ 76us) client_negotiation.cc:240] Negotiation successful
0928 14:42:50.287957 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260928 14:42:50.298148 11079 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:42:50.298270 11079 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:42:50.299062 11079 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4e38cb65782d4298acda01cec350da9f: Sending DeleteTablet for 3 replicas of tablet 81fa674baf7e4906bcd869cf71ee5ad3
I20260928 14:42:50.299396 11208 tablet_service.cc:1514] Processing DeleteTablet for tablet 81fa674baf7e4906bcd869cf71ee5ad3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:42:50 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:40144
I20260928 14:42:50.299456 11357 tablet_service.cc:1514] Processing DeleteTablet for tablet 81fa674baf7e4906bcd869cf71ee5ad3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:42:50 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:37502
I20260928 14:42:50.299803 11497 tablet_service.cc:1514] Processing DeleteTablet for tablet 81fa674baf7e4906bcd869cf71ee5ad3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:42:50 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:47546
I20260928 14:42:50.300014 11079 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46824:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260928 14:42:50.300088 11634 tablet_replica.cc:330] T 81fa674baf7e4906bcd869cf71ee5ad3 P 8f0bbb4eaedd49b9b5a0d487b3860300: stopping tablet replica
W20260928 14:42:50.300163 11079 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:42:50.300366 11634 raft_consensus.cc:2229] T 81fa674baf7e4906bcd869cf71ee5ad3 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:42:50.300550 11634 raft_consensus.cc:2258] T 81fa674baf7e4906bcd869cf71ee5ad3 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:42:50.300760 11633 tablet_replica.cc:330] T 81fa674baf7e4906bcd869cf71ee5ad3 P 6475771aee444d78832d41ed481d4255: stopping tablet replica
I20260928 14:42:50.300956 11633 raft_consensus.cc:2229] T 81fa674baf7e4906bcd869cf71ee5ad3 P 6475771aee444d78832d41ed481d4255 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:42:50.301062 11633 raft_consensus.cc:2258] T 81fa674baf7e4906bcd869cf71ee5ad3 P 6475771aee444d78832d41ed481d4255 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:42:50.301393 11497 tablet_service.cc:1467] Processing CreateTablet for tablet 7e1e42879b1f4f78bf4e873304322e73 (DEFAULT_TABLE table=test-table [id=2a8e687100694423b7675682fac6bdb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:42:50.301527 11497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e1e42879b1f4f78bf4e873304322e73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:50.301568 11208 tablet_service.cc:1467] Processing CreateTablet for tablet 7e1e42879b1f4f78bf4e873304322e73 (DEFAULT_TABLE table=test-table [id=2a8e687100694423b7675682fac6bdb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:42:50.301682 11208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e1e42879b1f4f78bf4e873304322e73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:50.302980 11587 tablet_bootstrap.cc:492] T 7e1e42879b1f4f78bf4e873304322e73 P 4cfa07b117ef40fca06be7d703b98707: Bootstrap starting.
I20260928 14:42:50.303004 11586 tablet_bootstrap.cc:492] T 7e1e42879b1f4f78bf4e873304322e73 P 6475771aee444d78832d41ed481d4255: Bootstrap starting.
I20260928 14:42:50.303409 11586 tablet_bootstrap.cc:654] T 7e1e42879b1f4f78bf4e873304322e73 P 6475771aee444d78832d41ed481d4255: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:50.303432 11587 tablet_bootstrap.cc:654] T 7e1e42879b1f4f78bf4e873304322e73 P 4cfa07b117ef40fca06be7d703b98707: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:50.303970 11357 tablet_service.cc:1467] Processing CreateTablet for tablet 7e1e42879b1f4f78bf4e873304322e73 (DEFAULT_TABLE table=test-table [id=2a8e687100694423b7675682fac6bdb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:42:50.304087 11357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e1e42879b1f4f78bf4e873304322e73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:50.305351 11588 tablet_bootstrap.cc:492] T 7e1e42879b1f4f78bf4e873304322e73 P 8f0bbb4eaedd49b9b5a0d487b3860300: Bootstrap starting.
I20260928 14:42:50.305792 11588 tablet_bootstrap.cc:654] T 7e1e42879b1f4f78bf4e873304322e73 P 8f0bbb4eaedd49b9b5a0d487b3860300: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:50.306156 11633 ts_tablet_manager.cc:1905] T 81fa674baf7e4906bcd869cf71ee5ad3 P 6475771aee444d78832d41ed481d4255: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:42:50.306473 11634 ts_tablet_manager.cc:1905] T 81fa674baf7e4906bcd869cf71ee5ad3 P 8f0bbb4eaedd49b9b5a0d487b3860300: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:42:50.308140 11634 ts_tablet_manager.cc:1918] T 81fa674baf7e4906bcd869cf71ee5ad3 P 8f0bbb4eaedd49b9b5a0d487b3860300: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:42:50.308175 11633 ts_tablet_manager.cc:1918] T 81fa674baf7e4906bcd869cf71ee5ad3 P 6475771aee444d78832d41ed481d4255: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:42:50.308243 11633 log.cc:1196] T 81fa674baf7e4906bcd869cf71ee5ad3 P 6475771aee444d78832d41ed481d4255: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-0/wal/wals/81fa674baf7e4906bcd869cf71ee5ad3
I20260928 14:42:50.308342 11635 tablet_replica.cc:330] T 81fa674baf7e4906bcd869cf71ee5ad3 P 4cfa07b117ef40fca06be7d703b98707: stopping tablet replica
I20260928 14:42:50.308358 11634 log.cc:1196] T 81fa674baf7e4906bcd869cf71ee5ad3 P 8f0bbb4eaedd49b9b5a0d487b3860300: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-1/wal/wals/81fa674baf7e4906bcd869cf71ee5ad3
I20260928 14:42:50.308468 11635 raft_consensus.cc:2229] T 81fa674baf7e4906bcd869cf71ee5ad3 P 4cfa07b117ef40fca06be7d703b98707 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:42:50.308545 11635 raft_consensus.cc:2258] T 81fa674baf7e4906bcd869cf71ee5ad3 P 4cfa07b117ef40fca06be7d703b98707 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:42:50.308565 11633 ts_tablet_manager.cc:1939] T 81fa674baf7e4906bcd869cf71ee5ad3 P 6475771aee444d78832d41ed481d4255: Deleting consensus metadata
I20260928 14:42:50.308950 11635 ts_tablet_manager.cc:1905] T 81fa674baf7e4906bcd869cf71ee5ad3 P 4cfa07b117ef40fca06be7d703b98707: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:42:50.308967 11634 ts_tablet_manager.cc:1939] T 81fa674baf7e4906bcd869cf71ee5ad3 P 8f0bbb4eaedd49b9b5a0d487b3860300: Deleting consensus metadata
I20260928 14:42:50.309005 11067 catalog_manager.cc:4841] TS 6475771aee444d78832d41ed481d4255 (127.9.33.65:46725): tablet 81fa674baf7e4906bcd869cf71ee5ad3 (table test-table [id=562f104592484bc9b04cf01baf1a3add]) successfully deleted
I20260928 14:42:50.309504 11066 catalog_manager.cc:4841] TS 8f0bbb4eaedd49b9b5a0d487b3860300 (127.9.33.66:36645): tablet 81fa674baf7e4906bcd869cf71ee5ad3 (table test-table [id=562f104592484bc9b04cf01baf1a3add]) successfully deleted
I20260928 14:42:50.309654 11586 tablet_bootstrap.cc:492] T 7e1e42879b1f4f78bf4e873304322e73 P 6475771aee444d78832d41ed481d4255: No bootstrap required, opened a new log
I20260928 14:42:50.309710 11586 ts_tablet_manager.cc:1397] T 7e1e42879b1f4f78bf4e873304322e73 P 6475771aee444d78832d41ed481d4255: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260928 14:42:50.309868 11586 raft_consensus.cc:348] T 7e1e42879b1f4f78bf4e873304322e73 P 6475771aee444d78832d41ed481d4255 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } }
I20260928 14:42:50.309940 11586 raft_consensus.cc:374] T 7e1e42879b1f4f78bf4e873304322e73 P 6475771aee444d78832d41ed481d4255 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:50.309960 11586 raft_consensus.cc:729] T 7e1e42879b1f4f78bf4e873304322e73 P 6475771aee444d78832d41ed481d4255 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6475771aee444d78832d41ed481d4255, State: Initialized, Role: FOLLOWER
I20260928 14:42:50.310001 11586 consensus_queue.cc:260] T 7e1e42879b1f4f78bf4e873304322e73 P 6475771aee444d78832d41ed481d4255 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } }
I20260928 14:42:50.310035 11588 tablet_bootstrap.cc:492] T 7e1e42879b1f4f78bf4e873304322e73 P 8f0bbb4eaedd49b9b5a0d487b3860300: No bootstrap required, opened a new log
I20260928 14:42:50.310081 11586 ts_tablet_manager.cc:1428] T 7e1e42879b1f4f78bf4e873304322e73 P 6475771aee444d78832d41ed481d4255: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260928 14:42:50.310078 11588 ts_tablet_manager.cc:1397] T 7e1e42879b1f4f78bf4e873304322e73 P 8f0bbb4eaedd49b9b5a0d487b3860300: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260928 14:42:50.310218 11588 raft_consensus.cc:348] T 7e1e42879b1f4f78bf4e873304322e73 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } }
I20260928 14:42:50.310293 11588 raft_consensus.cc:374] T 7e1e42879b1f4f78bf4e873304322e73 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:50.310314 11588 raft_consensus.cc:729] T 7e1e42879b1f4f78bf4e873304322e73 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f0bbb4eaedd49b9b5a0d487b3860300, State: Initialized, Role: FOLLOWER
I20260928 14:42:50.310353 11588 consensus_queue.cc:260] T 7e1e42879b1f4f78bf4e873304322e73 P 8f0bbb4eaedd49b9b5a0d487b3860300 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } }
I20260928 14:42:50.310453 11588 ts_tablet_manager.cc:1428] T 7e1e42879b1f4f78bf4e873304322e73 P 8f0bbb4eaedd49b9b5a0d487b3860300: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:50.310930 11587 tablet_bootstrap.cc:492] T 7e1e42879b1f4f78bf4e873304322e73 P 4cfa07b117ef40fca06be7d703b98707: No bootstrap required, opened a new log
I20260928 14:42:50.310977 11587 ts_tablet_manager.cc:1397] T 7e1e42879b1f4f78bf4e873304322e73 P 4cfa07b117ef40fca06be7d703b98707: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20260928 14:42:50.311082 11587 raft_consensus.cc:348] T 7e1e42879b1f4f78bf4e873304322e73 P 4cfa07b117ef40fca06be7d703b98707 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } }
I20260928 14:42:50.311149 11587 raft_consensus.cc:374] T 7e1e42879b1f4f78bf4e873304322e73 P 4cfa07b117ef40fca06be7d703b98707 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:50.311169 11587 raft_consensus.cc:729] T 7e1e42879b1f4f78bf4e873304322e73 P 4cfa07b117ef40fca06be7d703b98707 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cfa07b117ef40fca06be7d703b98707, State: Initialized, Role: FOLLOWER
I20260928 14:42:50.311210 11587 consensus_queue.cc:260] T 7e1e42879b1f4f78bf4e873304322e73 P 4cfa07b117ef40fca06be7d703b98707 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } }
I20260928 14:42:50.311371 11587 ts_tablet_manager.cc:1428] T 7e1e42879b1f4f78bf4e873304322e73 P 4cfa07b117ef40fca06be7d703b98707: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260928 14:42:50.311203 11635 ts_tablet_manager.cc:1918] T 81fa674baf7e4906bcd869cf71ee5ad3 P 4cfa07b117ef40fca06be7d703b98707: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:42:50.311599 11635 log.cc:1196] T 81fa674baf7e4906bcd869cf71ee5ad3 P 4cfa07b117ef40fca06be7d703b98707: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-2/wal/wals/81fa674baf7e4906bcd869cf71ee5ad3
I20260928 14:42:50.311899 11635 ts_tablet_manager.cc:1939] T 81fa674baf7e4906bcd869cf71ee5ad3 P 4cfa07b117ef40fca06be7d703b98707: Deleting consensus metadata
I20260928 14:42:50.312291 11064 catalog_manager.cc:4841] TS 4cfa07b117ef40fca06be7d703b98707 (127.9.33.67:39601): tablet 81fa674baf7e4906bcd869cf71ee5ad3 (table test-table [id=562f104592484bc9b04cf01baf1a3add]) successfully deleted
I20260928 14:42:50.601447 11631 raft_consensus.cc:482] T 7e1e42879b1f4f78bf4e873304322e73 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:42:50.601567 11631 raft_consensus.cc:504] T 7e1e42879b1f4f78bf4e873304322e73 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } }
I20260928 14:42:50.601794 11631 leader_election.cc:288] T 7e1e42879b1f4f78bf4e873304322e73 P 8f0bbb4eaedd49b9b5a0d487b3860300 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6475771aee444d78832d41ed481d4255 (127.9.33.65:46725), 4cfa07b117ef40fca06be7d703b98707 (127.9.33.67:39601)
I20260928 14:42:50.602018 11237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e1e42879b1f4f78bf4e873304322e73" candidate_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6475771aee444d78832d41ed481d4255" is_pre_election: true
I20260928 14:42:50.602018 11517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e1e42879b1f4f78bf4e873304322e73" candidate_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cfa07b117ef40fca06be7d703b98707" is_pre_election: true
I20260928 14:42:50.602133 11237 raft_consensus.cc:2454] T 7e1e42879b1f4f78bf4e873304322e73 P 6475771aee444d78832d41ed481d4255 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f0bbb4eaedd49b9b5a0d487b3860300 in term 0.
I20260928 14:42:50.602133 11517 raft_consensus.cc:2454] T 7e1e42879b1f4f78bf4e873304322e73 P 4cfa07b117ef40fca06be7d703b98707 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f0bbb4eaedd49b9b5a0d487b3860300 in term 0.
I20260928 14:42:50.602286 11308 leader_election.cc:302] T 7e1e42879b1f4f78bf4e873304322e73 P 8f0bbb4eaedd49b9b5a0d487b3860300 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cfa07b117ef40fca06be7d703b98707, 8f0bbb4eaedd49b9b5a0d487b3860300; no voters:
I20260928 14:42:50.602468 11631 raft_consensus.cc:2781] T 7e1e42879b1f4f78bf4e873304322e73 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:42:50.602512 11631 raft_consensus.cc:482] T 7e1e42879b1f4f78bf4e873304322e73 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:42:50.602526 11631 raft_consensus.cc:3037] T 7e1e42879b1f4f78bf4e873304322e73 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:50.603286 11631 raft_consensus.cc:504] T 7e1e42879b1f4f78bf4e873304322e73 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } }
I20260928 14:42:50.603432 11631 leader_election.cc:288] T 7e1e42879b1f4f78bf4e873304322e73 P 8f0bbb4eaedd49b9b5a0d487b3860300 [CANDIDATE]: Term 1 election: Requested vote from peers 6475771aee444d78832d41ed481d4255 (127.9.33.65:46725), 4cfa07b117ef40fca06be7d703b98707 (127.9.33.67:39601)
I20260928 14:42:50.603583 11237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e1e42879b1f4f78bf4e873304322e73" candidate_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6475771aee444d78832d41ed481d4255"
I20260928 14:42:50.603596 11517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e1e42879b1f4f78bf4e873304322e73" candidate_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cfa07b117ef40fca06be7d703b98707"
I20260928 14:42:50.603681 11237 raft_consensus.cc:3037] T 7e1e42879b1f4f78bf4e873304322e73 P 6475771aee444d78832d41ed481d4255 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:50.603679 11517 raft_consensus.cc:3037] T 7e1e42879b1f4f78bf4e873304322e73 P 4cfa07b117ef40fca06be7d703b98707 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:50.604318 11237 raft_consensus.cc:2454] T 7e1e42879b1f4f78bf4e873304322e73 P 6475771aee444d78832d41ed481d4255 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f0bbb4eaedd49b9b5a0d487b3860300 in term 1.
I20260928 14:42:50.604483 11517 raft_consensus.cc:2454] T 7e1e42879b1f4f78bf4e873304322e73 P 4cfa07b117ef40fca06be7d703b98707 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f0bbb4eaedd49b9b5a0d487b3860300 in term 1.
I20260928 14:42:50.604521 11311 leader_election.cc:302] T 7e1e42879b1f4f78bf4e873304322e73 P 8f0bbb4eaedd49b9b5a0d487b3860300 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6475771aee444d78832d41ed481d4255, 8f0bbb4eaedd49b9b5a0d487b3860300; no voters:
I20260928 14:42:50.604619 11631 raft_consensus.cc:2781] T 7e1e42879b1f4f78bf4e873304322e73 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:42:50.604667 11631 raft_consensus.cc:686] T 7e1e42879b1f4f78bf4e873304322e73 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 1 LEADER]: Becoming Leader. State: Replica: 8f0bbb4eaedd49b9b5a0d487b3860300, State: Running, Role: LEADER
I20260928 14:42:50.604724 11631 consensus_queue.cc:237] T 7e1e42879b1f4f78bf4e873304322e73 P 8f0bbb4eaedd49b9b5a0d487b3860300 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } }
I20260928 14:42:50.605404 11079 catalog_manager.cc:5495] T 7e1e42879b1f4f78bf4e873304322e73 P 8f0bbb4eaedd49b9b5a0d487b3860300 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f0bbb4eaedd49b9b5a0d487b3860300 (127.9.33.66). New cstate: current_term: 1 leader_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } health_report { overall_health: UNKNOWN } } }
I20260928 14:42:50.664325 11237 raft_consensus.cc:1261] T 7e1e42879b1f4f78bf4e873304322e73 P 6475771aee444d78832d41ed481d4255 [term 1 FOLLOWER]: Refusing update from remote peer 8f0bbb4eaedd49b9b5a0d487b3860300: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:42:50.664345 11517 raft_consensus.cc:1261] T 7e1e42879b1f4f78bf4e873304322e73 P 4cfa07b117ef40fca06be7d703b98707 [term 1 FOLLOWER]: Refusing update from remote peer 8f0bbb4eaedd49b9b5a0d487b3860300: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:42:50.664558 11594 consensus_queue.cc:1035] T 7e1e42879b1f4f78bf4e873304322e73 P 8f0bbb4eaedd49b9b5a0d487b3860300 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:42:50.664660 11631 consensus_queue.cc:1035] T 7e1e42879b1f4f78bf4e873304322e73 P 8f0bbb4eaedd49b9b5a0d487b3860300 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260928 14:42:50.668782 11615 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260928 14:42:52.186617 11079 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:42:52.186735 11079 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46824:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:42:52.187609 11079 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4e38cb65782d4298acda01cec350da9f: Sending DeleteTablet for 3 replicas of tablet 7e1e42879b1f4f78bf4e873304322e73
I20260928 14:42:52.187887 11208 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e1e42879b1f4f78bf4e873304322e73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:42:52 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:40144
I20260928 14:42:52.187904 11497 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e1e42879b1f4f78bf4e873304322e73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:42:52 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:47546
I20260928 14:42:52.187952 11357 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e1e42879b1f4f78bf4e873304322e73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:42:52 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:37502
I20260928 14:42:52.188180 11663 tablet_replica.cc:330] T 7e1e42879b1f4f78bf4e873304322e73 P 4cfa07b117ef40fca06be7d703b98707: stopping tablet replica
I20260928 14:42:52.188277 11663 raft_consensus.cc:2229] T 7e1e42879b1f4f78bf4e873304322e73 P 4cfa07b117ef40fca06be7d703b98707 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:42:52.188323 11663 raft_consensus.cc:2258] T 7e1e42879b1f4f78bf4e873304322e73 P 4cfa07b117ef40fca06be7d703b98707 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:42:52.188501 11664 tablet_replica.cc:330] T 7e1e42879b1f4f78bf4e873304322e73 P 6475771aee444d78832d41ed481d4255: stopping tablet replica
I20260928 14:42:52.188534 11663 ts_tablet_manager.cc:1905] T 7e1e42879b1f4f78bf4e873304322e73 P 4cfa07b117ef40fca06be7d703b98707: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:42:52.188568 11664 raft_consensus.cc:2229] T 7e1e42879b1f4f78bf4e873304322e73 P 6475771aee444d78832d41ed481d4255 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:42:52.188611 11664 raft_consensus.cc:2258] T 7e1e42879b1f4f78bf4e873304322e73 P 6475771aee444d78832d41ed481d4255 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:42:52.188709 11665 tablet_replica.cc:330] T 7e1e42879b1f4f78bf4e873304322e73 P 8f0bbb4eaedd49b9b5a0d487b3860300: stopping tablet replica
I20260928 14:42:52.188771 11665 raft_consensus.cc:2229] T 7e1e42879b1f4f78bf4e873304322e73 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:42:52.188843 11665 raft_consensus.cc:2258] T 7e1e42879b1f4f78bf4e873304322e73 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:42:52.188935 11664 ts_tablet_manager.cc:1905] T 7e1e42879b1f4f78bf4e873304322e73 P 6475771aee444d78832d41ed481d4255: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:42:52.189128 11665 ts_tablet_manager.cc:1905] T 7e1e42879b1f4f78bf4e873304322e73 P 8f0bbb4eaedd49b9b5a0d487b3860300: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:42:52.190865 11664 ts_tablet_manager.cc:1918] T 7e1e42879b1f4f78bf4e873304322e73 P 6475771aee444d78832d41ed481d4255: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 14:42:52.190941 11664 log.cc:1196] T 7e1e42879b1f4f78bf4e873304322e73 P 6475771aee444d78832d41ed481d4255: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-0/wal/wals/7e1e42879b1f4f78bf4e873304322e73
I20260928 14:42:52.191062 11665 ts_tablet_manager.cc:1918] T 7e1e42879b1f4f78bf4e873304322e73 P 8f0bbb4eaedd49b9b5a0d487b3860300: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 14:42:52.191121 11665 log.cc:1196] T 7e1e42879b1f4f78bf4e873304322e73 P 8f0bbb4eaedd49b9b5a0d487b3860300: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-1/wal/wals/7e1e42879b1f4f78bf4e873304322e73
I20260928 14:42:52.191205 11664 ts_tablet_manager.cc:1939] T 7e1e42879b1f4f78bf4e873304322e73 P 6475771aee444d78832d41ed481d4255: Deleting consensus metadata
I20260928 14:42:52.191383 11665 ts_tablet_manager.cc:1939] T 7e1e42879b1f4f78bf4e873304322e73 P 8f0bbb4eaedd49b9b5a0d487b3860300: Deleting consensus metadata
I20260928 14:42:52.191577 11067 catalog_manager.cc:4841] TS 6475771aee444d78832d41ed481d4255 (127.9.33.65:46725): tablet 7e1e42879b1f4f78bf4e873304322e73 (table test-table [id=2a8e687100694423b7675682fac6bdb9]) successfully deleted
I20260928 14:42:52.191756 11066 catalog_manager.cc:4841] TS 8f0bbb4eaedd49b9b5a0d487b3860300 (127.9.33.66:36645): tablet 7e1e42879b1f4f78bf4e873304322e73 (table test-table [id=2a8e687100694423b7675682fac6bdb9]) successfully deleted
I20260928 14:42:52.192903 11663 ts_tablet_manager.cc:1918] T 7e1e42879b1f4f78bf4e873304322e73 P 4cfa07b117ef40fca06be7d703b98707: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 14:42:52.192968 11663 log.cc:1196] T 7e1e42879b1f4f78bf4e873304322e73 P 4cfa07b117ef40fca06be7d703b98707: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-2/wal/wals/7e1e42879b1f4f78bf4e873304322e73
I20260928 14:42:52.193181 11663 ts_tablet_manager.cc:1939] T 7e1e42879b1f4f78bf4e873304322e73 P 4cfa07b117ef40fca06be7d703b98707: Deleting consensus metadata
I20260928 14:42:52.193555 11064 catalog_manager.cc:4841] TS 4cfa07b117ef40fca06be7d703b98707 (127.9.33.67:39601): tablet 7e1e42879b1f4f78bf4e873304322e73 (table test-table [id=2a8e687100694423b7675682fac6bdb9]) successfully deleted
Sep 28 14:42:52 dist-test-slave-pv0w krb5kdc[11035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606572, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 28 14:42:52 dist-test-slave-pv0w krb5kdc[11035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606572, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:42:52.218590 11673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:52.205513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46842 (local address 127.9.33.126:46127)
0928 14:42:52.205649 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:52.205651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:52.205660 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:42:52.205717 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:52.205718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:52.205728 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:52.205788 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:52.206357 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:52.206899 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:52.207641 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:52.208004 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:52.208059 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:52.216030 (+ 7971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:52.216032 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:52.216032 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:52.216040 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:52.217430 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:52.217778 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:52.217779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:52.217779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:52.217834 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:52.218064 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:52.218064 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:52.218064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:52.218174 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:52.218250 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:52.218384 (+ 134us) server_negotiation.cc:299] Negotiation successful
0928 14:42:52.218452 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":78,"threads_started":1}
I20260928 14:42:52.220093 11079 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46842:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:42:52.220244 11079 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:42:52.221625 11208 tablet_service.cc:1467] Processing CreateTablet for tablet d50a187f5542445bb0145f5dc51bf8dc (DEFAULT_TABLE table=test-table [id=cbad3f00b0fd404cb96d01fff8f3f2b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:42:52.221652 11497 tablet_service.cc:1467] Processing CreateTablet for tablet d50a187f5542445bb0145f5dc51bf8dc (DEFAULT_TABLE table=test-table [id=cbad3f00b0fd404cb96d01fff8f3f2b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:42:52.221652 11357 tablet_service.cc:1467] Processing CreateTablet for tablet d50a187f5542445bb0145f5dc51bf8dc (DEFAULT_TABLE table=test-table [id=cbad3f00b0fd404cb96d01fff8f3f2b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:42:52.221786 11497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d50a187f5542445bb0145f5dc51bf8dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:52.221786 11357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d50a187f5542445bb0145f5dc51bf8dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:52.221789 11208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d50a187f5542445bb0145f5dc51bf8dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:52.222822 11674 tablet_bootstrap.cc:492] T d50a187f5542445bb0145f5dc51bf8dc P 6475771aee444d78832d41ed481d4255: Bootstrap starting.
I20260928 14:42:52.223208 11675 tablet_bootstrap.cc:492] T d50a187f5542445bb0145f5dc51bf8dc P 4cfa07b117ef40fca06be7d703b98707: Bootstrap starting.
I20260928 14:42:52.223351 11674 tablet_bootstrap.cc:654] T d50a187f5542445bb0145f5dc51bf8dc P 6475771aee444d78832d41ed481d4255: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:52.223591 11675 tablet_bootstrap.cc:654] T d50a187f5542445bb0145f5dc51bf8dc P 4cfa07b117ef40fca06be7d703b98707: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:52.223968 11676 tablet_bootstrap.cc:492] T d50a187f5542445bb0145f5dc51bf8dc P 8f0bbb4eaedd49b9b5a0d487b3860300: Bootstrap starting.
I20260928 14:42:52.224107 11674 tablet_bootstrap.cc:492] T d50a187f5542445bb0145f5dc51bf8dc P 6475771aee444d78832d41ed481d4255: No bootstrap required, opened a new log
I20260928 14:42:52.224164 11674 ts_tablet_manager.cc:1397] T d50a187f5542445bb0145f5dc51bf8dc P 6475771aee444d78832d41ed481d4255: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 14:42:52.224356 11676 tablet_bootstrap.cc:654] T d50a187f5542445bb0145f5dc51bf8dc P 8f0bbb4eaedd49b9b5a0d487b3860300: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:52.224339 11674 raft_consensus.cc:348] T d50a187f5542445bb0145f5dc51bf8dc P 6475771aee444d78832d41ed481d4255 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } }
I20260928 14:42:52.224428 11674 raft_consensus.cc:374] T d50a187f5542445bb0145f5dc51bf8dc P 6475771aee444d78832d41ed481d4255 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:52.224459 11674 raft_consensus.cc:729] T d50a187f5542445bb0145f5dc51bf8dc P 6475771aee444d78832d41ed481d4255 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6475771aee444d78832d41ed481d4255, State: Initialized, Role: FOLLOWER
I20260928 14:42:52.224514 11674 consensus_queue.cc:260] T d50a187f5542445bb0145f5dc51bf8dc P 6475771aee444d78832d41ed481d4255 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } }
I20260928 14:42:52.224746 11674 ts_tablet_manager.cc:1428] T d50a187f5542445bb0145f5dc51bf8dc P 6475771aee444d78832d41ed481d4255: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260928 14:42:52.224788 11675 tablet_bootstrap.cc:492] T d50a187f5542445bb0145f5dc51bf8dc P 4cfa07b117ef40fca06be7d703b98707: No bootstrap required, opened a new log
I20260928 14:42:52.224854 11675 ts_tablet_manager.cc:1397] T d50a187f5542445bb0145f5dc51bf8dc P 4cfa07b117ef40fca06be7d703b98707: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260928 14:42:52.224972 11675 raft_consensus.cc:348] T d50a187f5542445bb0145f5dc51bf8dc P 4cfa07b117ef40fca06be7d703b98707 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } }
I20260928 14:42:52.225068 11675 raft_consensus.cc:374] T d50a187f5542445bb0145f5dc51bf8dc P 4cfa07b117ef40fca06be7d703b98707 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:52.225102 11675 raft_consensus.cc:729] T d50a187f5542445bb0145f5dc51bf8dc P 4cfa07b117ef40fca06be7d703b98707 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cfa07b117ef40fca06be7d703b98707, State: Initialized, Role: FOLLOWER
I20260928 14:42:52.225152 11675 consensus_queue.cc:260] T d50a187f5542445bb0145f5dc51bf8dc P 4cfa07b117ef40fca06be7d703b98707 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } }
I20260928 14:42:52.225299 11675 ts_tablet_manager.cc:1428] T d50a187f5542445bb0145f5dc51bf8dc P 4cfa07b117ef40fca06be7d703b98707: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260928 14:42:52.225843 11676 tablet_bootstrap.cc:492] T d50a187f5542445bb0145f5dc51bf8dc P 8f0bbb4eaedd49b9b5a0d487b3860300: No bootstrap required, opened a new log
I20260928 14:42:52.225947 11676 ts_tablet_manager.cc:1397] T d50a187f5542445bb0145f5dc51bf8dc P 8f0bbb4eaedd49b9b5a0d487b3860300: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:42:52.226075 11676 raft_consensus.cc:348] T d50a187f5542445bb0145f5dc51bf8dc P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } }
I20260928 14:42:52.226146 11676 raft_consensus.cc:374] T d50a187f5542445bb0145f5dc51bf8dc P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:52.226161 11676 raft_consensus.cc:729] T d50a187f5542445bb0145f5dc51bf8dc P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f0bbb4eaedd49b9b5a0d487b3860300, State: Initialized, Role: FOLLOWER
I20260928 14:42:52.226233 11676 consensus_queue.cc:260] T d50a187f5542445bb0145f5dc51bf8dc P 8f0bbb4eaedd49b9b5a0d487b3860300 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } }
I20260928 14:42:52.226359 11676 ts_tablet_manager.cc:1428] T d50a187f5542445bb0145f5dc51bf8dc P 8f0bbb4eaedd49b9b5a0d487b3860300: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:52.264285 11679 raft_consensus.cc:482] T d50a187f5542445bb0145f5dc51bf8dc P 6475771aee444d78832d41ed481d4255 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:42:52.264420 11679 raft_consensus.cc:504] T d50a187f5542445bb0145f5dc51bf8dc P 6475771aee444d78832d41ed481d4255 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } }
I20260928 14:42:52.264791 11679 leader_election.cc:288] T d50a187f5542445bb0145f5dc51bf8dc P 6475771aee444d78832d41ed481d4255 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8f0bbb4eaedd49b9b5a0d487b3860300 (127.9.33.66:36645), 4cfa07b117ef40fca06be7d703b98707 (127.9.33.67:39601)
I20260928 14:42:52.269166 11683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:52.265098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:59931 (local address 127.9.33.66:36645)
0928 14:42:52.265283 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:52.265285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:52.265294 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:42:52.265395 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:52.265395 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:52.265408 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:52.265499 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:42:52.265713 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:52.266528 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:52.268138 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:52.268786 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:52.268866 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:52.268880 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:52.269045 (+ 165us) server_negotiation.cc:299] Negotiation successful
0928 14:42:52.269087 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":101,"threads_started":1}
I20260928 14:42:52.269166 11684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:52.265098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:39749 (local address 127.9.33.67:39601)
0928 14:42:52.265361 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:52.265363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:52.265395 (+ 32us) server_negotiation.cc:407] Connection header received
0928 14:42:52.265479 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:52.265480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:52.265491 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:52.265561 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:42:52.265713 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:52.266503 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:52.268272 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:52.268810 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:52.268873 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:52.268912 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:52.269045 (+ 133us) server_negotiation.cc:299] Negotiation successful
0928 14:42:52.269087 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":105,"threads_started":1}
I20260928 14:42:52.269227 11681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:52.265009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:36645 (local address 127.9.33.65:59931)
0928 14:42:52.265226 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:52.265239 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:52.265300 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:52.265512 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:52.265513 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:52.265519 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:42:52.265627 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:52.265630 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:52.266611 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:52.266613 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:52.268044 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:52.268048 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:52.268873 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:52.268874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:52.268959 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:52.268964 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:42:52.269025 (+ 61us) client_negotiation.cc:240] Negotiation successful
0928 14:42:52.269080 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":4,"thread_start_us":122,"threads_started":1}
I20260928 14:42:52.269227 11682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:52.265009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:39601 (local address 127.9.33.65:39749)
0928 14:42:52.265337 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:52.265347 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:52.265409 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:52.265577 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:52.265577 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:52.265579 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:42:52.265638 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:52.265639 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:52.266611 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:52.266613 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:52.268190 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:52.268192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:52.268877 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:52.268878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:52.268962 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:52.268964 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 14:42:52.269025 (+ 61us) client_negotiation.cc:240] Negotiation successful
0928 14:42:52.269080 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"mutex_wait_us":56,"thread_start_us":121,"threads_started":1}
I20260928 14:42:52.269505 11377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d50a187f5542445bb0145f5dc51bf8dc" candidate_uuid: "6475771aee444d78832d41ed481d4255" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" is_pre_election: true
I20260928 14:42:52.269608 11377 raft_consensus.cc:2454] T d50a187f5542445bb0145f5dc51bf8dc P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6475771aee444d78832d41ed481d4255 in term 0.
I20260928 14:42:52.269629 11517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d50a187f5542445bb0145f5dc51bf8dc" candidate_uuid: "6475771aee444d78832d41ed481d4255" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cfa07b117ef40fca06be7d703b98707" is_pre_election: true
I20260928 14:42:52.269749 11517 raft_consensus.cc:2454] T d50a187f5542445bb0145f5dc51bf8dc P 4cfa07b117ef40fca06be7d703b98707 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6475771aee444d78832d41ed481d4255 in term 0.
I20260928 14:42:52.269832 11170 leader_election.cc:302] T d50a187f5542445bb0145f5dc51bf8dc P 6475771aee444d78832d41ed481d4255 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6475771aee444d78832d41ed481d4255, 8f0bbb4eaedd49b9b5a0d487b3860300; no voters:
I20260928 14:42:52.269972 11679 raft_consensus.cc:2781] T d50a187f5542445bb0145f5dc51bf8dc P 6475771aee444d78832d41ed481d4255 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:42:52.270018 11679 raft_consensus.cc:482] T d50a187f5542445bb0145f5dc51bf8dc P 6475771aee444d78832d41ed481d4255 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:42:52.270051 11679 raft_consensus.cc:3037] T d50a187f5542445bb0145f5dc51bf8dc P 6475771aee444d78832d41ed481d4255 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:52.270738 11679 raft_consensus.cc:504] T d50a187f5542445bb0145f5dc51bf8dc P 6475771aee444d78832d41ed481d4255 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } }
I20260928 14:42:52.270902 11679 leader_election.cc:288] T d50a187f5542445bb0145f5dc51bf8dc P 6475771aee444d78832d41ed481d4255 [CANDIDATE]: Term 1 election: Requested vote from peers 8f0bbb4eaedd49b9b5a0d487b3860300 (127.9.33.66:36645), 4cfa07b117ef40fca06be7d703b98707 (127.9.33.67:39601)
I20260928 14:42:52.271064 11377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d50a187f5542445bb0145f5dc51bf8dc" candidate_uuid: "6475771aee444d78832d41ed481d4255" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300"
I20260928 14:42:52.271159 11377 raft_consensus.cc:3037] T d50a187f5542445bb0145f5dc51bf8dc P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:52.271133 11517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d50a187f5542445bb0145f5dc51bf8dc" candidate_uuid: "6475771aee444d78832d41ed481d4255" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cfa07b117ef40fca06be7d703b98707"
I20260928 14:42:52.271238 11517 raft_consensus.cc:3037] T d50a187f5542445bb0145f5dc51bf8dc P 4cfa07b117ef40fca06be7d703b98707 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:52.271869 11377 raft_consensus.cc:2454] T d50a187f5542445bb0145f5dc51bf8dc P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6475771aee444d78832d41ed481d4255 in term 1.
I20260928 14:42:52.271903 11517 raft_consensus.cc:2454] T d50a187f5542445bb0145f5dc51bf8dc P 4cfa07b117ef40fca06be7d703b98707 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6475771aee444d78832d41ed481d4255 in term 1.
I20260928 14:42:52.272070 11168 leader_election.cc:302] T d50a187f5542445bb0145f5dc51bf8dc P 6475771aee444d78832d41ed481d4255 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cfa07b117ef40fca06be7d703b98707, 6475771aee444d78832d41ed481d4255; no voters:
I20260928 14:42:52.272184 11679 raft_consensus.cc:2781] T d50a187f5542445bb0145f5dc51bf8dc P 6475771aee444d78832d41ed481d4255 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:42:52.272374 11679 raft_consensus.cc:686] T d50a187f5542445bb0145f5dc51bf8dc P 6475771aee444d78832d41ed481d4255 [term 1 LEADER]: Becoming Leader. State: Replica: 6475771aee444d78832d41ed481d4255, State: Running, Role: LEADER
I20260928 14:42:52.272467 11679 consensus_queue.cc:237] T d50a187f5542445bb0145f5dc51bf8dc P 6475771aee444d78832d41ed481d4255 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } }
I20260928 14:42:52.273183 11080 catalog_manager.cc:5495] T d50a187f5542445bb0145f5dc51bf8dc P 6475771aee444d78832d41ed481d4255 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6475771aee444d78832d41ed481d4255 (127.9.33.65). New cstate: current_term: 1 leader_uuid: "6475771aee444d78832d41ed481d4255" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } health_report { overall_health: UNKNOWN } } }
I20260928 14:42:52.286715 11688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:52.284331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40150 (local address 127.9.33.65:46725)
0928 14:42:52.284485 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:52.284487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:52.284495 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:42:52.284539 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:52.284539 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:52.284556 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:52.284626 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:42:52.284737 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:52.285225 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:52.285976 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:52.286182 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:52.286233 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:52.286380 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:42:52.286487 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:42:52.286542 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:52.286578 (+ 36us) server_negotiation.cc:299] Negotiation successful
0928 14:42:52.286594 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":99,"threads_started":1}
I20260928 14:42:52.291651 11684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:52.289132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:53113 (local address 127.9.33.67:39601)
0928 14:42:52.289175 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:52.289177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:52.289294 (+ 117us) server_negotiation.cc:407] Connection header received
0928 14:42:52.289343 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:52.289343 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:52.289353 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:52.289398 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:42:52.289536 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:52.290018 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:52.290854 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:52.291131 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:52.291202 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:52.291397 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:42:52.291430 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:42:52.291481 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:52.291553 (+ 72us) server_negotiation.cc:299] Negotiation successful
0928 14:42:52.291574 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 14:42:52.291671 11691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:52.289030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:39601 (local address 127.9.33.65:53113)
0928 14:42:52.289243 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:52.289252 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:52.289303 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:52.289401 (+ 98us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:52.289402 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:52.289403 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 14:42:52.289451 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:52.289452 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:52.290084 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:52.290085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:52.290756 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:52.290758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:52.291213 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:52.291214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:52.291282 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:52.291296 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 14:42:52.291495 (+ 199us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 14:42:52.291497 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 14:42:52.291546 (+ 49us) client_negotiation.cc:240] Negotiation successful
0928 14:42:52.291591 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":116,"threads_started":1}
I20260928 14:42:52.293875 11517 raft_consensus.cc:1261] T d50a187f5542445bb0145f5dc51bf8dc P 4cfa07b117ef40fca06be7d703b98707 [term 1 FOLLOWER]: Refusing update from remote peer 6475771aee444d78832d41ed481d4255: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:42:52.293875 11377 raft_consensus.cc:1261] T d50a187f5542445bb0145f5dc51bf8dc P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 1 FOLLOWER]: Refusing update from remote peer 6475771aee444d78832d41ed481d4255: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:42:52.294142 11685 consensus_queue.cc:1035] T d50a187f5542445bb0145f5dc51bf8dc P 6475771aee444d78832d41ed481d4255 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:42:52.294265 11679 consensus_queue.cc:1035] T d50a187f5542445bb0145f5dc51bf8dc P 6475771aee444d78832d41ed481d4255 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:42:52.302050 11688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:52.299556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:55101 (local address 127.9.33.65:46725)
0928 14:42:52.299601 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:52.299603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:52.299708 (+ 105us) server_negotiation.cc:407] Connection header received
0928 14:42:52.299794 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:52.299794 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:52.299812 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:52.299862 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:42:52.299955 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:52.300426 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:52.301341 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:52.301656 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:52.301756 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:52.301874 (+ 118us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:42:52.301902 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:42:52.301939 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:52.301974 (+ 35us) server_negotiation.cc:299] Negotiation successful
0928 14:42:52.301992 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 14:42:52.302071 11696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:52.299497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:46725 (local address 127.9.33.67:55101)
0928 14:42:52.299631 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:52.299641 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:52.299743 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:52.299868 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:52.299869 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:52.299870 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 14:42:52.299919 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:52.299921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:52.300508 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:52.300509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:52.301219 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:52.301221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:52.301735 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:52.301736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:52.301805 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:52.301811 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 14:42:52.301940 (+ 129us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 14:42:52.301941 (+ 1us) client_negotiation.cc:769] Sending connection context
0928 14:42:52.301972 (+ 31us) client_negotiation.cc:240] Negotiation successful
0928 14:42:52.302011 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":79,"threads_started":1}
I20260928 14:42:52.308465 11079 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:42:52.308584 11079 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:42:52.309257 11079 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4e38cb65782d4298acda01cec350da9f: Sending DeleteTablet for 3 replicas of tablet d50a187f5542445bb0145f5dc51bf8dc
I20260928 14:42:52.309553 11497 tablet_service.cc:1514] Processing DeleteTablet for tablet d50a187f5542445bb0145f5dc51bf8dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:42:52 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:47546
I20260928 14:42:52.309571 11208 tablet_service.cc:1514] Processing DeleteTablet for tablet d50a187f5542445bb0145f5dc51bf8dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:42:52 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:40144
I20260928 14:42:52.309651 11663 tablet_replica.cc:330] T d50a187f5542445bb0145f5dc51bf8dc P 4cfa07b117ef40fca06be7d703b98707: stopping tablet replica
I20260928 14:42:52.309659 11664 tablet_replica.cc:330] T d50a187f5542445bb0145f5dc51bf8dc P 6475771aee444d78832d41ed481d4255: stopping tablet replica
I20260928 14:42:52.309705 11663 raft_consensus.cc:2229] T d50a187f5542445bb0145f5dc51bf8dc P 4cfa07b117ef40fca06be7d703b98707 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:42:52.309716 11664 raft_consensus.cc:2229] T d50a187f5542445bb0145f5dc51bf8dc P 6475771aee444d78832d41ed481d4255 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:42:52.309742 11663 raft_consensus.cc:2258] T d50a187f5542445bb0145f5dc51bf8dc P 4cfa07b117ef40fca06be7d703b98707 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:42:52.309670 11357 tablet_service.cc:1514] Processing DeleteTablet for tablet d50a187f5542445bb0145f5dc51bf8dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:42:52 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:37502
I20260928 14:42:52.309777 11664 raft_consensus.cc:2258] T d50a187f5542445bb0145f5dc51bf8dc P 6475771aee444d78832d41ed481d4255 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:42:52.309863 11665 tablet_replica.cc:330] T d50a187f5542445bb0145f5dc51bf8dc P 8f0bbb4eaedd49b9b5a0d487b3860300: stopping tablet replica
I20260928 14:42:52.309926 11665 raft_consensus.cc:2229] T d50a187f5542445bb0145f5dc51bf8dc P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:42:52.309967 11665 raft_consensus.cc:2258] T d50a187f5542445bb0145f5dc51bf8dc P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:42:52.310099 11663 ts_tablet_manager.cc:1905] T d50a187f5542445bb0145f5dc51bf8dc P 4cfa07b117ef40fca06be7d703b98707: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:42:52.310254 11664 ts_tablet_manager.cc:1905] T d50a187f5542445bb0145f5dc51bf8dc P 6475771aee444d78832d41ed481d4255: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:42:52.310320 11665 ts_tablet_manager.cc:1905] T d50a187f5542445bb0145f5dc51bf8dc P 8f0bbb4eaedd49b9b5a0d487b3860300: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:42:52.311865 11663 ts_tablet_manager.cc:1918] T d50a187f5542445bb0145f5dc51bf8dc P 4cfa07b117ef40fca06be7d703b98707: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:42:52.311934 11663 log.cc:1196] T d50a187f5542445bb0145f5dc51bf8dc P 4cfa07b117ef40fca06be7d703b98707: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-2/wal/wals/d50a187f5542445bb0145f5dc51bf8dc
I20260928 14:42:52.312047 11079 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46842:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260928 14:42:52.312167 11663 ts_tablet_manager.cc:1939] T d50a187f5542445bb0145f5dc51bf8dc P 4cfa07b117ef40fca06be7d703b98707: Deleting consensus metadata
W20260928 14:42:52.312245 11079 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:42:52.312507 11665 ts_tablet_manager.cc:1918] T d50a187f5542445bb0145f5dc51bf8dc P 8f0bbb4eaedd49b9b5a0d487b3860300: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:42:52.312565 11665 log.cc:1196] T d50a187f5542445bb0145f5dc51bf8dc P 8f0bbb4eaedd49b9b5a0d487b3860300: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-1/wal/wals/d50a187f5542445bb0145f5dc51bf8dc
I20260928 14:42:52.312687 11064 catalog_manager.cc:4841] TS 4cfa07b117ef40fca06be7d703b98707 (127.9.33.67:39601): tablet d50a187f5542445bb0145f5dc51bf8dc (table test-table [id=cbad3f00b0fd404cb96d01fff8f3f2b5]) successfully deleted
I20260928 14:42:52.312798 11665 ts_tablet_manager.cc:1939] T d50a187f5542445bb0145f5dc51bf8dc P 8f0bbb4eaedd49b9b5a0d487b3860300: Deleting consensus metadata
I20260928 14:42:52.313095 11066 catalog_manager.cc:4841] TS 8f0bbb4eaedd49b9b5a0d487b3860300 (127.9.33.66:36645): tablet d50a187f5542445bb0145f5dc51bf8dc (table test-table [id=cbad3f00b0fd404cb96d01fff8f3f2b5]) successfully deleted
I20260928 14:42:52.313165 11664 ts_tablet_manager.cc:1918] T d50a187f5542445bb0145f5dc51bf8dc P 6475771aee444d78832d41ed481d4255: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:42:52.313230 11664 log.cc:1196] T d50a187f5542445bb0145f5dc51bf8dc P 6475771aee444d78832d41ed481d4255: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-0/wal/wals/d50a187f5542445bb0145f5dc51bf8dc
I20260928 14:42:52.313449 11664 ts_tablet_manager.cc:1939] T d50a187f5542445bb0145f5dc51bf8dc P 6475771aee444d78832d41ed481d4255: Deleting consensus metadata
I20260928 14:42:52.313757 11067 catalog_manager.cc:4841] TS 6475771aee444d78832d41ed481d4255 (127.9.33.65:46725): tablet d50a187f5542445bb0145f5dc51bf8dc (table test-table [id=cbad3f00b0fd404cb96d01fff8f3f2b5]) successfully deleted
I20260928 14:42:52.313827 11497 tablet_service.cc:1467] Processing CreateTablet for tablet cf922ec3a4e048288c3a2ee263488210 (DEFAULT_TABLE table=test-table [id=77effdb4e8d846f6819ef9363d535c67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:42:52.313833 11208 tablet_service.cc:1467] Processing CreateTablet for tablet cf922ec3a4e048288c3a2ee263488210 (DEFAULT_TABLE table=test-table [id=77effdb4e8d846f6819ef9363d535c67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:42:52.313933 11497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf922ec3a4e048288c3a2ee263488210. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:52.313941 11208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf922ec3a4e048288c3a2ee263488210. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:52.314851 11675 tablet_bootstrap.cc:492] T cf922ec3a4e048288c3a2ee263488210 P 4cfa07b117ef40fca06be7d703b98707: Bootstrap starting.
I20260928 14:42:52.314940 11357 tablet_service.cc:1467] Processing CreateTablet for tablet cf922ec3a4e048288c3a2ee263488210 (DEFAULT_TABLE table=test-table [id=77effdb4e8d846f6819ef9363d535c67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:42:52.315043 11357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf922ec3a4e048288c3a2ee263488210. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:52.315140 11674 tablet_bootstrap.cc:492] T cf922ec3a4e048288c3a2ee263488210 P 6475771aee444d78832d41ed481d4255: Bootstrap starting.
I20260928 14:42:52.315234 11675 tablet_bootstrap.cc:654] T cf922ec3a4e048288c3a2ee263488210 P 4cfa07b117ef40fca06be7d703b98707: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:52.315536 11674 tablet_bootstrap.cc:654] T cf922ec3a4e048288c3a2ee263488210 P 6475771aee444d78832d41ed481d4255: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:52.315832 11675 tablet_bootstrap.cc:492] T cf922ec3a4e048288c3a2ee263488210 P 4cfa07b117ef40fca06be7d703b98707: No bootstrap required, opened a new log
I20260928 14:42:52.315884 11675 ts_tablet_manager.cc:1397] T cf922ec3a4e048288c3a2ee263488210 P 4cfa07b117ef40fca06be7d703b98707: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260928 14:42:52.316005 11675 raft_consensus.cc:348] T cf922ec3a4e048288c3a2ee263488210 P 4cfa07b117ef40fca06be7d703b98707 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } }
I20260928 14:42:52.316085 11675 raft_consensus.cc:374] T cf922ec3a4e048288c3a2ee263488210 P 4cfa07b117ef40fca06be7d703b98707 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:52.316107 11675 raft_consensus.cc:729] T cf922ec3a4e048288c3a2ee263488210 P 4cfa07b117ef40fca06be7d703b98707 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cfa07b117ef40fca06be7d703b98707, State: Initialized, Role: FOLLOWER
I20260928 14:42:52.316161 11675 consensus_queue.cc:260] T cf922ec3a4e048288c3a2ee263488210 P 4cfa07b117ef40fca06be7d703b98707 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } }
I20260928 14:42:52.316268 11675 ts_tablet_manager.cc:1428] T cf922ec3a4e048288c3a2ee263488210 P 4cfa07b117ef40fca06be7d703b98707: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:52.316431 11676 tablet_bootstrap.cc:492] T cf922ec3a4e048288c3a2ee263488210 P 8f0bbb4eaedd49b9b5a0d487b3860300: Bootstrap starting.
I20260928 14:42:52.316748 11676 tablet_bootstrap.cc:654] T cf922ec3a4e048288c3a2ee263488210 P 8f0bbb4eaedd49b9b5a0d487b3860300: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:52.316926 11674 tablet_bootstrap.cc:492] T cf922ec3a4e048288c3a2ee263488210 P 6475771aee444d78832d41ed481d4255: No bootstrap required, opened a new log
I20260928 14:42:52.316972 11674 ts_tablet_manager.cc:1397] T cf922ec3a4e048288c3a2ee263488210 P 6475771aee444d78832d41ed481d4255: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:42:52.317133 11674 raft_consensus.cc:348] T cf922ec3a4e048288c3a2ee263488210 P 6475771aee444d78832d41ed481d4255 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } }
I20260928 14:42:52.317195 11674 raft_consensus.cc:374] T cf922ec3a4e048288c3a2ee263488210 P 6475771aee444d78832d41ed481d4255 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:52.317216 11674 raft_consensus.cc:729] T cf922ec3a4e048288c3a2ee263488210 P 6475771aee444d78832d41ed481d4255 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6475771aee444d78832d41ed481d4255, State: Initialized, Role: FOLLOWER
I20260928 14:42:52.317267 11674 consensus_queue.cc:260] T cf922ec3a4e048288c3a2ee263488210 P 6475771aee444d78832d41ed481d4255 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } }
I20260928 14:42:52.317322 11676 tablet_bootstrap.cc:492] T cf922ec3a4e048288c3a2ee263488210 P 8f0bbb4eaedd49b9b5a0d487b3860300: No bootstrap required, opened a new log
I20260928 14:42:52.317368 11676 ts_tablet_manager.cc:1397] T cf922ec3a4e048288c3a2ee263488210 P 8f0bbb4eaedd49b9b5a0d487b3860300: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 14:42:52.317436 11674 ts_tablet_manager.cc:1428] T cf922ec3a4e048288c3a2ee263488210 P 6475771aee444d78832d41ed481d4255: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:52.317515 11676 raft_consensus.cc:348] T cf922ec3a4e048288c3a2ee263488210 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } }
I20260928 14:42:52.317584 11676 raft_consensus.cc:374] T cf922ec3a4e048288c3a2ee263488210 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:52.317603 11676 raft_consensus.cc:729] T cf922ec3a4e048288c3a2ee263488210 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f0bbb4eaedd49b9b5a0d487b3860300, State: Initialized, Role: FOLLOWER
I20260928 14:42:52.317675 11676 consensus_queue.cc:260] T cf922ec3a4e048288c3a2ee263488210 P 8f0bbb4eaedd49b9b5a0d487b3860300 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } }
I20260928 14:42:52.317778 11676 ts_tablet_manager.cc:1428] T cf922ec3a4e048288c3a2ee263488210 P 8f0bbb4eaedd49b9b5a0d487b3860300: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:52.373010 11654 raft_consensus.cc:482] T cf922ec3a4e048288c3a2ee263488210 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:42:52.373129 11654 raft_consensus.cc:504] T cf922ec3a4e048288c3a2ee263488210 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } }
I20260928 14:42:52.373311 11654 leader_election.cc:288] T cf922ec3a4e048288c3a2ee263488210 P 8f0bbb4eaedd49b9b5a0d487b3860300 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6475771aee444d78832d41ed481d4255 (127.9.33.65:46725), 4cfa07b117ef40fca06be7d703b98707 (127.9.33.67:39601)
I20260928 14:42:52.373541 11517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf922ec3a4e048288c3a2ee263488210" candidate_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cfa07b117ef40fca06be7d703b98707" is_pre_election: true
I20260928 14:42:52.373541 11237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf922ec3a4e048288c3a2ee263488210" candidate_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6475771aee444d78832d41ed481d4255" is_pre_election: true
I20260928 14:42:52.373651 11237 raft_consensus.cc:2454] T cf922ec3a4e048288c3a2ee263488210 P 6475771aee444d78832d41ed481d4255 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f0bbb4eaedd49b9b5a0d487b3860300 in term 0.
I20260928 14:42:52.373651 11517 raft_consensus.cc:2454] T cf922ec3a4e048288c3a2ee263488210 P 4cfa07b117ef40fca06be7d703b98707 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f0bbb4eaedd49b9b5a0d487b3860300 in term 0.
I20260928 14:42:52.373801 11308 leader_election.cc:302] T cf922ec3a4e048288c3a2ee263488210 P 8f0bbb4eaedd49b9b5a0d487b3860300 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4cfa07b117ef40fca06be7d703b98707, 6475771aee444d78832d41ed481d4255, 8f0bbb4eaedd49b9b5a0d487b3860300; no voters:
I20260928 14:42:52.373898 11654 raft_consensus.cc:2781] T cf922ec3a4e048288c3a2ee263488210 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:42:52.373934 11654 raft_consensus.cc:482] T cf922ec3a4e048288c3a2ee263488210 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:42:52.373953 11654 raft_consensus.cc:3037] T cf922ec3a4e048288c3a2ee263488210 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:52.374773 11654 raft_consensus.cc:504] T cf922ec3a4e048288c3a2ee263488210 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } }
I20260928 14:42:52.374927 11654 leader_election.cc:288] T cf922ec3a4e048288c3a2ee263488210 P 8f0bbb4eaedd49b9b5a0d487b3860300 [CANDIDATE]: Term 1 election: Requested vote from peers 6475771aee444d78832d41ed481d4255 (127.9.33.65:46725), 4cfa07b117ef40fca06be7d703b98707 (127.9.33.67:39601)
I20260928 14:42:52.375077 11237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf922ec3a4e048288c3a2ee263488210" candidate_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6475771aee444d78832d41ed481d4255"
I20260928 14:42:52.375154 11237 raft_consensus.cc:3037] T cf922ec3a4e048288c3a2ee263488210 P 6475771aee444d78832d41ed481d4255 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:52.375147 11517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf922ec3a4e048288c3a2ee263488210" candidate_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cfa07b117ef40fca06be7d703b98707"
I20260928 14:42:52.375240 11517 raft_consensus.cc:3037] T cf922ec3a4e048288c3a2ee263488210 P 4cfa07b117ef40fca06be7d703b98707 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:52.375833 11237 raft_consensus.cc:2454] T cf922ec3a4e048288c3a2ee263488210 P 6475771aee444d78832d41ed481d4255 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f0bbb4eaedd49b9b5a0d487b3860300 in term 1.
I20260928 14:42:52.375995 11311 leader_election.cc:302] T cf922ec3a4e048288c3a2ee263488210 P 8f0bbb4eaedd49b9b5a0d487b3860300 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6475771aee444d78832d41ed481d4255, 8f0bbb4eaedd49b9b5a0d487b3860300; no voters:
I20260928 14:42:52.376109 11654 raft_consensus.cc:2781] T cf922ec3a4e048288c3a2ee263488210 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:42:52.376102 11517 raft_consensus.cc:2454] T cf922ec3a4e048288c3a2ee263488210 P 4cfa07b117ef40fca06be7d703b98707 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f0bbb4eaedd49b9b5a0d487b3860300 in term 1.
I20260928 14:42:52.376147 11654 raft_consensus.cc:686] T cf922ec3a4e048288c3a2ee263488210 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 1 LEADER]: Becoming Leader. State: Replica: 8f0bbb4eaedd49b9b5a0d487b3860300, State: Running, Role: LEADER
I20260928 14:42:52.376195 11654 consensus_queue.cc:237] T cf922ec3a4e048288c3a2ee263488210 P 8f0bbb4eaedd49b9b5a0d487b3860300 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } }
I20260928 14:42:52.376771 11079 catalog_manager.cc:5495] T cf922ec3a4e048288c3a2ee263488210 P 8f0bbb4eaedd49b9b5a0d487b3860300 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f0bbb4eaedd49b9b5a0d487b3860300 (127.9.33.66). New cstate: current_term: 1 leader_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f0bbb4eaedd49b9b5a0d487b3860300" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36645 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 } health_report { overall_health: UNKNOWN } } }
I20260928 14:42:52.395526 11683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:52.393215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37510 (local address 127.9.33.66:36645)
0928 14:42:52.393263 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:52.393265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:52.393274 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:42:52.393361 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:52.393362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:52.393375 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:52.393429 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:42:52.393547 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:52.394061 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:52.394815 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:52.395026 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:52.395103 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:52.395258 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:42:52.395300 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:42:52.395362 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:52.395414 (+ 52us) server_negotiation.cc:299] Negotiation successful
0928 14:42:52.395440 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 14:42:52.398092 11517 raft_consensus.cc:1261] T cf922ec3a4e048288c3a2ee263488210 P 4cfa07b117ef40fca06be7d703b98707 [term 1 FOLLOWER]: Refusing update from remote peer 8f0bbb4eaedd49b9b5a0d487b3860300: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:42:52.398159 11237 raft_consensus.cc:1261] T cf922ec3a4e048288c3a2ee263488210 P 6475771aee444d78832d41ed481d4255 [term 1 FOLLOWER]: Refusing update from remote peer 8f0bbb4eaedd49b9b5a0d487b3860300: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:42:52.398334 11654 consensus_queue.cc:1035] T cf922ec3a4e048288c3a2ee263488210 P 8f0bbb4eaedd49b9b5a0d487b3860300 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cfa07b117ef40fca06be7d703b98707" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:42:52.398468 11654 consensus_queue.cc:1035] T cf922ec3a4e048288c3a2ee263488210 P 8f0bbb4eaedd49b9b5a0d487b3860300 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6475771aee444d78832d41ed481d4255" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 46725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260928 14:42:52.403266 11645 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260928 14:42:53.921663 11079 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:42:53.921774 11079 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:42:53.922549 11079 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4e38cb65782d4298acda01cec350da9f: Sending DeleteTablet for 3 replicas of tablet cf922ec3a4e048288c3a2ee263488210
I20260928 14:42:53.922873 11357 tablet_service.cc:1514] Processing DeleteTablet for tablet cf922ec3a4e048288c3a2ee263488210 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:42:53 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:37502
I20260928 14:42:53.922912 11208 tablet_service.cc:1514] Processing DeleteTablet for tablet cf922ec3a4e048288c3a2ee263488210 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:42:53 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:40144
I20260928 14:42:53.922966 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 11152
I20260928 14:42:53.924458 11497 tablet_service.cc:1514] Processing DeleteTablet for tablet cf922ec3a4e048288c3a2ee263488210 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:42:53 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:47546
I20260928 14:42:53.924649 11729 tablet_replica.cc:330] T cf922ec3a4e048288c3a2ee263488210 P 4cfa07b117ef40fca06be7d703b98707: stopping tablet replica
I20260928 14:42:53.924738 11729 raft_consensus.cc:2229] T cf922ec3a4e048288c3a2ee263488210 P 4cfa07b117ef40fca06be7d703b98707 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:42:53.924800 11728 tablet_replica.cc:330] T cf922ec3a4e048288c3a2ee263488210 P 8f0bbb4eaedd49b9b5a0d487b3860300: stopping tablet replica
I20260928 14:42:53.924810 11729 raft_consensus.cc:2258] T cf922ec3a4e048288c3a2ee263488210 P 4cfa07b117ef40fca06be7d703b98707 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:42:53.924865 11728 raft_consensus.cc:2229] T cf922ec3a4e048288c3a2ee263488210 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:42:53.924947 11728 raft_consensus.cc:2258] T cf922ec3a4e048288c3a2ee263488210 P 8f0bbb4eaedd49b9b5a0d487b3860300 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:42:53.925156 11728 ts_tablet_manager.cc:1905] T cf922ec3a4e048288c3a2ee263488210 P 8f0bbb4eaedd49b9b5a0d487b3860300: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:42:53.925158 11729 ts_tablet_manager.cc:1905] T cf922ec3a4e048288c3a2ee263488210 P 4cfa07b117ef40fca06be7d703b98707: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:42:53.926964 11728 ts_tablet_manager.cc:1918] T cf922ec3a4e048288c3a2ee263488210 P 8f0bbb4eaedd49b9b5a0d487b3860300: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 14:42:53.926964 11729 ts_tablet_manager.cc:1918] T cf922ec3a4e048288c3a2ee263488210 P 4cfa07b117ef40fca06be7d703b98707: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260928 14:42:53.927031 11728 log.cc:1196] T cf922ec3a4e048288c3a2ee263488210 P 8f0bbb4eaedd49b9b5a0d487b3860300: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-1/wal/wals/cf922ec3a4e048288c3a2ee263488210
I20260928 14:42:53.927155 11729 log.cc:1196] T cf922ec3a4e048288c3a2ee263488210 P 4cfa07b117ef40fca06be7d703b98707: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790606561545179-9349-0/minicluster-data/ts-2/wal/wals/cf922ec3a4e048288c3a2ee263488210
I20260928 14:42:53.927312 11728 ts_tablet_manager.cc:1939] T cf922ec3a4e048288c3a2ee263488210 P 8f0bbb4eaedd49b9b5a0d487b3860300: Deleting consensus metadata
I20260928 14:42:53.927491 11729 ts_tablet_manager.cc:1939] T cf922ec3a4e048288c3a2ee263488210 P 4cfa07b117ef40fca06be7d703b98707: Deleting consensus metadata
I20260928 14:42:53.927714 11066 catalog_manager.cc:4841] TS 8f0bbb4eaedd49b9b5a0d487b3860300 (127.9.33.66:36645): tablet cf922ec3a4e048288c3a2ee263488210 (table test-table [id=77effdb4e8d846f6819ef9363d535c67]) successfully deleted
I20260928 14:42:53.928023 11064 catalog_manager.cc:4841] TS 4cfa07b117ef40fca06be7d703b98707 (127.9.33.67:39601): tablet cf922ec3a4e048288c3a2ee263488210 (table test-table [id=77effdb4e8d846f6819ef9363d535c67]) successfully deleted
W20260928 14:42:53.929234 11067 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.33.65:46725 (error 108)
I20260928 14:42:53.929840 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 11292
I20260928 14:42:53.929852 11731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:53.929641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:46725 (local address 127.0.0.1:40164)
0928 14:42:53.929762 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:53.929816 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.65:46725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":78,"threads_started":1}
W20260928 14:42:53.935961 11067 catalog_manager.cc:4584] TS 6475771aee444d78832d41ed481d4255 (127.9.33.65:46725): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cf922ec3a4e048288c3a2ee263488210: Network error: Client connection negotiation failed: client connection to 127.9.33.65:46725: connect: Connection refused (error 111)
I20260928 14:42:53.936609 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 11432
I20260928 14:42:53.942468 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 11048
2026-09-28T14:42:53Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6013 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:42:54 dist-test-slave-pv0w krb5kdc[11734](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:42:54 dist-test-slave-pv0w krb5kdc[11734](info): set up 2 sockets
Sep 28 14:42:54 dist-test-slave-pv0w krb5kdc[11734](info): commencing operation
krb5kdc: starting...
W20260928 14:42:55.000420 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:42:55 dist-test-slave-pv0w krb5kdc[11734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:42:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:42:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.33.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
I20260928 14:42:55.123607 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:45851
--webserver_interface=127.9.33.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:33335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:45851
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:55.206423 11747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:55.206691 11747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:55.206730 11747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:55.208036 11747 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:42:55.208099 11747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:55.208127 11747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:55.208160 11747 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:42:55.208190 11747 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:42:55.209721 11747 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:33335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:45851
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:45851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:55.210011 11747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:55.210246 11747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:55.215210 11753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:55.215232 11752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:55.215265 11755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:55.215467 11747 server_base.cc:1034] running on GCE node
I20260928 14:42:55.215698 11747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:55.216033 11747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:55.217188 11747 hybrid_clock.cc:648] HybridClock initialized: now 1790606575217172 us; error 31 us; skew 500 ppm
Sep 28 14:42:55 dist-test-slave-pv0w krb5kdc[11734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606575, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:55.220017 11747 init.cc:377] Logged in from keytab as kudu/127.9.33.126@KRBTEST.COM (short username kudu)
I20260928 14:42:55.220460 11747 webserver.cc:460] Webserver started at http://127.9.33.126:34991/ using document root <none> and password file <none>
I20260928 14:42:55.220695 11747 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:55.220749 11747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:55.220935 11747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:55.222080 11747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "27606a944247412d9a4a87f44e35015d"
format_stamp: "Formatted at 2026-09-28 14:42:55 on dist-test-slave-pv0w"
server_key: "85c9c6e690d0f5e52e4256f41e5093c8"
server_key_iv: "0929eb2050fd2d6dc705bd4270fc1d17"
server_key_version: "encryptionkey@0"
I20260928 14:42:55.222471 11747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "27606a944247412d9a4a87f44e35015d"
format_stamp: "Formatted at 2026-09-28 14:42:55 on dist-test-slave-pv0w"
server_key: "85c9c6e690d0f5e52e4256f41e5093c8"
server_key_iv: "0929eb2050fd2d6dc705bd4270fc1d17"
server_key_version: "encryptionkey@0"
I20260928 14:42:55.223928 11747 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:42:55.224705 11762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:55.224887 11747 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:42:55.224998 11747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "27606a944247412d9a4a87f44e35015d"
format_stamp: "Formatted at 2026-09-28 14:42:55 on dist-test-slave-pv0w"
server_key: "85c9c6e690d0f5e52e4256f41e5093c8"
server_key_iv: "0929eb2050fd2d6dc705bd4270fc1d17"
server_key_version: "encryptionkey@0"
I20260928 14:42:55.225076 11747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:55.241904 11747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:55.244122 11747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:55.244311 11747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:55.248525 11747 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:45851
I20260928 14:42:55.248600 11814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:45851 every 8 connection(s)
I20260928 14:42:55.248977 11747 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:42:55.249820 11815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:55.252306 11815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27606a944247412d9a4a87f44e35015d: Bootstrap starting.
I20260928 14:42:55.252929 11815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27606a944247412d9a4a87f44e35015d: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:55.253234 11815 log.cc:824] T 00000000000000000000000000000000 P 27606a944247412d9a4a87f44e35015d: Log is configured to *not* fsync() on all Append() calls
I20260928 14:42:55.253952 11815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27606a944247412d9a4a87f44e35015d: No bootstrap required, opened a new log
I20260928 14:42:55.255546 11815 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27606a944247412d9a4a87f44e35015d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27606a944247412d9a4a87f44e35015d" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 45851 } }
I20260928 14:42:55.255712 11815 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27606a944247412d9a4a87f44e35015d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:55.255818 11815 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27606a944247412d9a4a87f44e35015d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27606a944247412d9a4a87f44e35015d, State: Initialized, Role: FOLLOWER
I20260928 14:42:55.256011 11815 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27606a944247412d9a4a87f44e35015d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27606a944247412d9a4a87f44e35015d" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 45851 } }
I20260928 14:42:55.256120 11815 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27606a944247412d9a4a87f44e35015d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:42:55.256166 11815 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27606a944247412d9a4a87f44e35015d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:42:55.256222 11815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27606a944247412d9a4a87f44e35015d [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:55.256747 11815 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27606a944247412d9a4a87f44e35015d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27606a944247412d9a4a87f44e35015d" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 45851 } }
I20260928 14:42:55.256870 11815 leader_election.cc:302] T 00000000000000000000000000000000 P 27606a944247412d9a4a87f44e35015d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27606a944247412d9a4a87f44e35015d; no voters:
I20260928 14:42:55.257071 11815 leader_election.cc:288] T 00000000000000000000000000000000 P 27606a944247412d9a4a87f44e35015d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:42:55.257123 11818 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27606a944247412d9a4a87f44e35015d [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:42:55.257336 11815 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27606a944247412d9a4a87f44e35015d [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:42:55.257526 11818 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27606a944247412d9a4a87f44e35015d [term 1 LEADER]: Becoming Leader. State: Replica: 27606a944247412d9a4a87f44e35015d, State: Running, Role: LEADER
I20260928 14:42:55.257678 11818 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27606a944247412d9a4a87f44e35015d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27606a944247412d9a4a87f44e35015d" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 45851 } }
I20260928 14:42:55.258097 11819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27606a944247412d9a4a87f44e35015d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27606a944247412d9a4a87f44e35015d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27606a944247412d9a4a87f44e35015d" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 45851 } } }
I20260928 14:42:55.258180 11819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27606a944247412d9a4a87f44e35015d [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:55.258358 11820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27606a944247412d9a4a87f44e35015d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27606a944247412d9a4a87f44e35015d. Latest consensus state: current_term: 1 leader_uuid: "27606a944247412d9a4a87f44e35015d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27606a944247412d9a4a87f44e35015d" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 45851 } } }
I20260928 14:42:55.258426 11820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27606a944247412d9a4a87f44e35015d [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:55.258423 11827 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:42:55.258997 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 11747
I20260928 14:42:55.259100 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:42:55.259282 9349 external_mini_cluster.cc:1428] Setting key afe3ecccbafadfcf04687cde347ab9e2
I20260928 14:42:55.259150 11827 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:42:55.261181 11827 catalog_manager.cc:1347] Generated new cluster ID: 58d7946e11d64ed28a1e621d7db6f5a2
I20260928 14:42:55.261235 11827 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 14:42:55 dist-test-slave-pv0w krb5kdc[11734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:42:55.269105 11827 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:42:55.269784 11827 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:42:55.272794 11837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:55.259806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37538 (local address 127.9.33.126:45851)
0928 14:42:55.260470 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:55.260479 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:55.260516 (+ 37us) server_negotiation.cc:407] Connection header received
0928 14:42:55.260633 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:55.260641 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:55.260698 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:55.260845 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:55.264461 (+ 3616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:55.265536 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:55.266323 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:55.266779 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:55.266907 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:55.269521 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:55.269525 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:55.269525 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:55.269541 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:55.271191 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:55.271577 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:55.271582 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:55.271582 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:55.271676 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:55.271921 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:55.271921 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:55.271922 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:55.272139 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:55.272317 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:55.272424 (+ 107us) server_negotiation.cc:299] Negotiation successful
0928 14:42:55.272590 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":90,"threads_started":1}
I20260928 14:42:55.274560 11827 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27606a944247412d9a4a87f44e35015d: Generated new TSK 0
I20260928 14:42:55.274736 11827 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.33.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab.
Entry for principal kudu/127.9.33.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab.
I20260928 14:42:55.311571 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.65:0
--local_ip_for_outbound_sockets=127.9.33.65
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:45851
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:33335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:55.414409 11841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:55.414625 11841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:55.414651 11841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:55.415966 11841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:55.416013 11841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:55.416042 11841 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.65
I20260928 14:42:55.417622 11841 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:33335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:45851
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.65
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:55.417902 11841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:55.418146 11841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:55.424021 11847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:55.424055 11846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:55.424022 11849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:55.424070 11841 server_base.cc:1034] running on GCE node
I20260928 14:42:55.424443 11841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:55.424654 11841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:55.425817 11841 hybrid_clock.cc:648] HybridClock initialized: now 1790606575425801 us; error 29 us; skew 500 ppm
Sep 28 14:42:55 dist-test-slave-pv0w krb5kdc[11734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606575, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:55.428459 11841 init.cc:377] Logged in from keytab as kudu/127.9.33.65@KRBTEST.COM (short username kudu)
I20260928 14:42:55.429004 11841 webserver.cc:460] Webserver started at http://127.9.33.65:38491/ using document root <none> and password file <none>
I20260928 14:42:55.429198 11841 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:55.429239 11841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:55.429332 11841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:55.430352 11841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-0/data/instance:
uuid: "ee491cf9a17a4c79a46d40db9220830b"
format_stamp: "Formatted at 2026-09-28 14:42:55 on dist-test-slave-pv0w"
server_key: "639b4d4ac3eaf34e91068f83c7c5ce35"
server_key_iv: "9bc706d427081d07485b84aa56080cae"
server_key_version: "encryptionkey@0"
I20260928 14:42:55.430667 11841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance:
uuid: "ee491cf9a17a4c79a46d40db9220830b"
format_stamp: "Formatted at 2026-09-28 14:42:55 on dist-test-slave-pv0w"
server_key: "639b4d4ac3eaf34e91068f83c7c5ce35"
server_key_iv: "9bc706d427081d07485b84aa56080cae"
server_key_version: "encryptionkey@0"
I20260928 14:42:55.432047 11841 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260928 14:42:55.432811 11856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:55.432992 11841 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:42:55.433082 11841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-0/wal
uuid: "ee491cf9a17a4c79a46d40db9220830b"
format_stamp: "Formatted at 2026-09-28 14:42:55 on dist-test-slave-pv0w"
server_key: "639b4d4ac3eaf34e91068f83c7c5ce35"
server_key_iv: "9bc706d427081d07485b84aa56080cae"
server_key_version: "encryptionkey@0"
I20260928 14:42:55.433156 11841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:55.444710 11841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:55.446795 11841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:55.446961 11841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:55.447227 11841 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:42:55.447557 11841 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:42:55.447590 11841 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:55.447613 11841 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:42:55.447661 11841 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:55.453625 11841 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.65:33911
I20260928 14:42:55.453662 11969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.65:33911 every 8 connection(s)
I20260928 14:42:55.454073 11841 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
I20260928 14:42:55.457724 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 11841
I20260928 14:42:55.457814 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance
I20260928 14:42:55.458053 9349 external_mini_cluster.cc:1428] Setting key 49b16760e9c0d964bb2ca5a9edefe41f
Sep 28 14:42:55 dist-test-slave-pv0w krb5kdc[11734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606575, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:42:55.464504 11837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:55.454925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:36235 (local address 127.9.33.126:45851)
0928 14:42:55.455027 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:55.455029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:55.455135 (+ 106us) server_negotiation.cc:407] Connection header received
0928 14:42:55.455648 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:55.455649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:55.455662 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:55.455711 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:55.456765 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:55.457304 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:55.458854 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:55.459100 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:55.459207 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:55.461480 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:55.461481 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:55.461482 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:55.461490 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:55.463001 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:55.463446 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:55.463448 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:55.463448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:55.463510 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:55.463809 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:55.463811 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:55.463811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:55.463933 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:55.464021 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:55.464323 (+ 302us) server_negotiation.cc:299] Negotiation successful
0928 14:42:55.464415 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260928 14:42:55.464696 11972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:55.454847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:45851 (local address 127.9.33.65:36235)
0928 14:42:55.455025 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:55.455069 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:55.455559 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:55.455796 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:55.455799 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:55.456437 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:55.456648 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:55.456654 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:55.457441 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:55.457442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:55.458771 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:55.458774 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:55.459289 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:55.459290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:55.459412 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:55.459990 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:55.459999 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:55.461383 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:55.463108 (+ 1725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:55.463116 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:55.463118 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:55.463353 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:55.463581 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:55.463582 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:55.463582 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:55.463696 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:55.464024 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:55.464032 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:55.464179 (+ 147us) client_negotiation.cc:769] Sending connection context
0928 14:42:55.464287 (+ 108us) client_negotiation.cc:240] Negotiation successful
0928 14:42:55.464449 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":73,"threads_started":1}
I20260928 14:42:55.465256 11970 heartbeater.cc:347] Connected to a master server at 127.9.33.126:45851
I20260928 14:42:55.465363 11970 heartbeater.cc:464] Registering TS with master...
I20260928 14:42:55.465579 11970 heartbeater.cc:511] Master 127.9.33.126:45851 requested a full tablet report, sending...
I20260928 14:42:55.466162 11779 ts_manager.cc:194] Registered new tserver with Master: ee491cf9a17a4c79a46d40db9220830b (127.9.33.65:33911)
WARNING: no policy specified for kudu/127.9.33.66@KRBTEST.COM; defaulting to no policy
I20260928 14:42:55.466835 11779 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.65@KRBTEST.COM'} at 127.9.33.65:36235
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab.
Entry for principal kudu/127.9.33.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab.
I20260928 14:42:55.482736 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.66:0
--local_ip_for_outbound_sockets=127.9.33.66
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:45851
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:33335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:55.567466 11975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:55.567669 11975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:55.567695 11975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:55.568943 11975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:55.568986 11975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:55.569015 11975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.66
I20260928 14:42:55.570421 11975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:33335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:45851
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.66
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:55.570680 11975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:55.570916 11975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:55.575996 11981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:55.576023 11975 server_base.cc:1034] running on GCE node
W20260928 14:42:55.576047 11980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:55.576164 11983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:55.576422 11975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:55.576673 11975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:55.577831 11975 hybrid_clock.cc:648] HybridClock initialized: now 1790606575577829 us; error 38 us; skew 500 ppm
Sep 28 14:42:55 dist-test-slave-pv0w krb5kdc[11734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606575, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:55.580289 11975 init.cc:377] Logged in from keytab as kudu/127.9.33.66@KRBTEST.COM (short username kudu)
I20260928 14:42:55.580762 11975 webserver.cc:460] Webserver started at http://127.9.33.66:36597/ using document root <none> and password file <none>
I20260928 14:42:55.580986 11975 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:55.581053 11975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:55.581204 11975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:55.582232 11975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-1/data/instance:
uuid: "b29acbcf9204468abaf520609a985da1"
format_stamp: "Formatted at 2026-09-28 14:42:55 on dist-test-slave-pv0w"
server_key: "3b81c1a22e37571ccdfd4e82af3ae88c"
server_key_iv: "5ee2044ba6a7efa60636f7399beb0611"
server_key_version: "encryptionkey@0"
I20260928 14:42:55.582607 11975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance:
uuid: "b29acbcf9204468abaf520609a985da1"
format_stamp: "Formatted at 2026-09-28 14:42:55 on dist-test-slave-pv0w"
server_key: "3b81c1a22e37571ccdfd4e82af3ae88c"
server_key_iv: "5ee2044ba6a7efa60636f7399beb0611"
server_key_version: "encryptionkey@0"
I20260928 14:42:55.583979 11975 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260928 14:42:55.584697 11990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:55.584878 11975 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:55.584977 11975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-1/wal
uuid: "b29acbcf9204468abaf520609a985da1"
format_stamp: "Formatted at 2026-09-28 14:42:55 on dist-test-slave-pv0w"
server_key: "3b81c1a22e37571ccdfd4e82af3ae88c"
server_key_iv: "5ee2044ba6a7efa60636f7399beb0611"
server_key_version: "encryptionkey@0"
I20260928 14:42:55.585060 11975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:55.596042 11975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:55.598291 11975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:55.598412 11975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:55.598587 11975 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:42:55.598829 11975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:42:55.598860 11975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:55.598882 11975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:42:55.598894 11975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:55.604735 11975 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.66:37795
I20260928 14:42:55.604780 12103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.66:37795 every 8 connection(s)
I20260928 14:42:55.605113 11975 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
I20260928 14:42:55.607831 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 11975
I20260928 14:42:55.607918 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance
I20260928 14:42:55.608111 9349 external_mini_cluster.cc:1428] Setting key 11abeb88041d7d36e7d764a88510c2a6
Sep 28 14:42:55 dist-test-slave-pv0w krb5kdc[11734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606575, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:42:55.616003 11837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:55.605877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:53677 (local address 127.9.33.126:45851)
0928 14:42:55.605946 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:55.605949 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:55.606140 (+ 191us) server_negotiation.cc:407] Connection header received
0928 14:42:55.606698 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:55.606699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:55.606713 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:55.606781 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:55.607825 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:55.608576 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:55.610135 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:55.610450 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:55.610531 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:55.612854 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:55.612856 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:55.612856 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:55.612864 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:55.614556 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:55.614967 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:55.614969 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:55.614969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:55.615043 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:55.615288 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:55.615290 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:55.615290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:55.615447 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:55.615523 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:55.615785 (+ 262us) server_negotiation.cc:299] Negotiation successful
0928 14:42:55.615902 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 14:42:55.616112 12106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:55.605828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:45851 (local address 127.9.33.66:53677)
0928 14:42:55.605995 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:55.606048 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:55.606582 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:55.606857 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:55.606861 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:55.607464 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:55.607688 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:55.607693 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:55.608666 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:55.608667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:55.610037 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:55.610041 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:55.610543 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:55.610544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:55.610657 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:55.611244 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:55.611252 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:55.612756 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:55.614646 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:55.614653 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:55.614655 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:55.614888 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:55.615118 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:55.615118 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:55.615118 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:55.615217 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:55.615516 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:55.615521 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:55.615639 (+ 118us) client_negotiation.cc:769] Sending connection context
0928 14:42:55.615765 (+ 126us) client_negotiation.cc:240] Negotiation successful
0928 14:42:55.615912 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":74,"threads_started":1}
I20260928 14:42:55.616704 12104 heartbeater.cc:347] Connected to a master server at 127.9.33.126:45851
I20260928 14:42:55.616830 12104 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.9.33.67@KRBTEST.COM; defaulting to no policy
I20260928 14:42:55.617038 12104 heartbeater.cc:511] Master 127.9.33.126:45851 requested a full tablet report, sending...
I20260928 14:42:55.617450 11779 ts_manager.cc:194] Registered new tserver with Master: b29acbcf9204468abaf520609a985da1 (127.9.33.66:37795)
I20260928 14:42:55.617843 11779 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.66@KRBTEST.COM'} at 127.9.33.66:53677
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab.
Entry for principal kudu/127.9.33.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab.
I20260928 14:42:55.635521 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.67:0
--local_ip_for_outbound_sockets=127.9.33.67
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:45851
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:33335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:55.716528 12109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:55.716733 12109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:55.716758 12109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:55.717942 12109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:55.717986 12109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:55.718014 12109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.67
I20260928 14:42:55.719461 12109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:33335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:45851
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.67
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:55.719731 12109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:55.720000 12109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 14:42:55.725093 12109 server_base.cc:1034] running on GCE node
W20260928 14:42:55.725078 12114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:55.725078 12115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:55.725101 12117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:55.725616 12109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:55.725871 12109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:55.727020 12109 hybrid_clock.cc:648] HybridClock initialized: now 1790606575727000 us; error 36 us; skew 500 ppm
Sep 28 14:42:55 dist-test-slave-pv0w krb5kdc[11734](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606575, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:55.729773 12109 init.cc:377] Logged in from keytab as kudu/127.9.33.67@KRBTEST.COM (short username kudu)
I20260928 14:42:55.730337 12109 webserver.cc:460] Webserver started at http://127.9.33.67:34487/ using document root <none> and password file <none>
I20260928 14:42:55.730587 12109 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:55.730659 12109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:55.730818 12109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:55.732018 12109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-2/data/instance:
uuid: "13f9ba9384554f99a1372bb8cb8526f2"
format_stamp: "Formatted at 2026-09-28 14:42:55 on dist-test-slave-pv0w"
server_key: "424f2cf5dcdadaa0e33b154b1258f4b6"
server_key_iv: "5b2f34a47364b7aa9aad8b883f6f6253"
server_key_version: "encryptionkey@0"
I20260928 14:42:55.732390 12109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance:
uuid: "13f9ba9384554f99a1372bb8cb8526f2"
format_stamp: "Formatted at 2026-09-28 14:42:55 on dist-test-slave-pv0w"
server_key: "424f2cf5dcdadaa0e33b154b1258f4b6"
server_key_iv: "5b2f34a47364b7aa9aad8b883f6f6253"
server_key_version: "encryptionkey@0"
I20260928 14:42:55.733952 12109 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260928 14:42:55.734876 12124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:55.735096 12109 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:42:55.735203 12109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-2/wal
uuid: "13f9ba9384554f99a1372bb8cb8526f2"
format_stamp: "Formatted at 2026-09-28 14:42:55 on dist-test-slave-pv0w"
server_key: "424f2cf5dcdadaa0e33b154b1258f4b6"
server_key_iv: "5b2f34a47364b7aa9aad8b883f6f6253"
server_key_version: "encryptionkey@0"
I20260928 14:42:55.735317 12109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:55.755283 12109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:55.757508 12109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:55.757678 12109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:55.758023 12109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:42:55.758392 12109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:42:55.758450 12109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:55.758494 12109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:42:55.758522 12109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:55.764587 12109 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.67:41657
I20260928 14:42:55.764633 12237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.67:41657 every 8 connection(s)
I20260928 14:42:55.765018 12109 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
I20260928 14:42:55.770776 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 12109
Sep 28 14:42:55 dist-test-slave-pv0w krb5kdc[11734](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606575, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:42:55.770869 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance
I20260928 14:42:55.771070 9349 external_mini_cluster.cc:1428] Setting key 686506dff6f0f08ac9113f613872de9c
I20260928 14:42:55.774637 11837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:55.765810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:45617 (local address 127.9.33.126:45851)
0928 14:42:55.765869 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:55.765871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:55.766050 (+ 179us) server_negotiation.cc:407] Connection header received
0928 14:42:55.766520 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:55.766521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:55.766531 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:55.766590 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:55.767491 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:55.768063 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:55.769074 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:55.769258 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:55.769306 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:55.771441 (+ 2135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:55.771443 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:55.771444 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:55.771452 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:55.772928 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:55.773344 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:55.773345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:55.773346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:55.773417 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:55.773711 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:55.773713 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:55.773713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:55.773892 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:55.773968 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:55.774370 (+ 402us) server_negotiation.cc:299] Negotiation successful
0928 14:42:55.774508 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260928 14:42:55.774887 12240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:55.765778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:45851 (local address 127.9.33.67:45617)
0928 14:42:55.765925 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:55.765967 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:55.766439 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:55.766643 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:55.766647 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:55.767206 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:55.767411 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:55.767416 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:55.768147 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:55.768148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:55.768998 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:55.769001 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:55.769327 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:55.769327 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:55.769428 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:55.769862 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:55.769870 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:55.771335 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:55.773014 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:55.773021 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:55.773023 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:55.773251 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:55.773483 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:55.773484 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:55.773484 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:55.773614 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:55.773998 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:55.774004 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:55.774190 (+ 186us) client_negotiation.cc:769] Sending connection context
0928 14:42:55.774307 (+ 117us) client_negotiation.cc:240] Negotiation successful
0928 14:42:55.774627 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":75,"threads_started":1}
I20260928 14:42:55.775415 12238 heartbeater.cc:347] Connected to a master server at 127.9.33.126:45851
I20260928 14:42:55.775543 12238 heartbeater.cc:464] Registering TS with master...
I20260928 14:42:55.775764 12238 heartbeater.cc:511] Master 127.9.33.126:45851 requested a full tablet report, sending...
I20260928 14:42:55.776206 11779 ts_manager.cc:194] Registered new tserver with Master: 13f9ba9384554f99a1372bb8cb8526f2 (127.9.33.67:41657)
I20260928 14:42:55.776765 11779 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.67@KRBTEST.COM'} at 127.9.33.67:45617
I20260928 14:42:55.783016 9349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260928 14:42:55.794032 11837 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:42:55.793058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33492 (local address 127.9.33.126:45851)
0928 14:42:55.793150 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:55.793152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:55.793162 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:42:55.793246 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:55.793246 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:55.793258 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:55.793322 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:55.793955 (+ 633us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33492: BlockingRecv error: recv got EOF from 127.0.0.1:33492 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260928 14:42:55.794879 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 11841
I20260928 14:42:55.799520 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 11975
I20260928 14:42:55.803606 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 12109
I20260928 14:42:55.807672 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 11747
2026-09-28T14:42:55Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1863 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:42:55 dist-test-slave-pv0w krb5kdc[12250](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:42:55 dist-test-slave-pv0w krb5kdc[12250](info): set up 2 sockets
Sep 28 14:42:55 dist-test-slave-pv0w krb5kdc[12250](info): commencing operation
krb5kdc: starting...
W20260928 14:42:56.852293 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:42:56 dist-test-slave-pv0w krb5kdc[12250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:42:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:42:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.33.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
I20260928 14:42:56.972518 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:37467
--webserver_interface=127.9.33.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:38771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:37467
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:57.053990 12263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:57.054203 12263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:57.054226 12263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:57.055418 12263 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:42:57.055456 12263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:57.055470 12263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:57.055485 12263 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:42:57.055532 12263 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:42:57.057257 12263 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:38771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:37467
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:37467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:57.057516 12263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:57.057740 12263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:57.062781 12271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:57.062845 12269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:57.062901 12263 server_base.cc:1034] running on GCE node
W20260928 14:42:57.062896 12268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:57.063297 12263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:57.063612 12263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:57.064781 12263 hybrid_clock.cc:648] HybridClock initialized: now 1790606577064771 us; error 40 us; skew 500 ppm
Sep 28 14:42:57 dist-test-slave-pv0w krb5kdc[12250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606577, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:57.067345 12263 init.cc:377] Logged in from keytab as kudu/127.9.33.126@KRBTEST.COM (short username kudu)
I20260928 14:42:57.067862 12263 webserver.cc:460] Webserver started at http://127.9.33.126:43179/ using document root <none> and password file <none>
I20260928 14:42:57.068099 12263 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:57.068166 12263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:57.068322 12263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:57.069422 12263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "1eac4563fb094bc19e51e5e39cdf7c5b"
format_stamp: "Formatted at 2026-09-28 14:42:57 on dist-test-slave-pv0w"
server_key: "37818f9f905567c966a88ea29aa318cd"
server_key_iv: "90d80d94ab9e9de296027bb867a459eb"
server_key_version: "encryptionkey@0"
I20260928 14:42:57.069797 12263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "1eac4563fb094bc19e51e5e39cdf7c5b"
format_stamp: "Formatted at 2026-09-28 14:42:57 on dist-test-slave-pv0w"
server_key: "37818f9f905567c966a88ea29aa318cd"
server_key_iv: "90d80d94ab9e9de296027bb867a459eb"
server_key_version: "encryptionkey@0"
I20260928 14:42:57.071158 12263 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:42:57.071970 12278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:57.072142 12263 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:42:57.072245 12263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "1eac4563fb094bc19e51e5e39cdf7c5b"
format_stamp: "Formatted at 2026-09-28 14:42:57 on dist-test-slave-pv0w"
server_key: "37818f9f905567c966a88ea29aa318cd"
server_key_iv: "90d80d94ab9e9de296027bb867a459eb"
server_key_version: "encryptionkey@0"
I20260928 14:42:57.072328 12263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:57.082656 12263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:57.085109 12263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:57.085285 12263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:57.089324 12263 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:37467
I20260928 14:42:57.089356 12330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:37467 every 8 connection(s)
I20260928 14:42:57.089728 12263 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:42:57.090626 12331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:57.093120 12331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1eac4563fb094bc19e51e5e39cdf7c5b: Bootstrap starting.
I20260928 14:42:57.093750 12331 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1eac4563fb094bc19e51e5e39cdf7c5b: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:57.094063 12331 log.cc:824] T 00000000000000000000000000000000 P 1eac4563fb094bc19e51e5e39cdf7c5b: Log is configured to *not* fsync() on all Append() calls
I20260928 14:42:57.094811 12331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1eac4563fb094bc19e51e5e39cdf7c5b: No bootstrap required, opened a new log
I20260928 14:42:57.096484 12331 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1eac4563fb094bc19e51e5e39cdf7c5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eac4563fb094bc19e51e5e39cdf7c5b" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37467 } }
I20260928 14:42:57.096622 12331 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1eac4563fb094bc19e51e5e39cdf7c5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:57.096647 12331 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1eac4563fb094bc19e51e5e39cdf7c5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1eac4563fb094bc19e51e5e39cdf7c5b, State: Initialized, Role: FOLLOWER
I20260928 14:42:57.096836 12331 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1eac4563fb094bc19e51e5e39cdf7c5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eac4563fb094bc19e51e5e39cdf7c5b" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37467 } }
I20260928 14:42:57.096900 12331 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1eac4563fb094bc19e51e5e39cdf7c5b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:42:57.096922 12331 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1eac4563fb094bc19e51e5e39cdf7c5b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:42:57.096949 12331 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1eac4563fb094bc19e51e5e39cdf7c5b [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:57.097311 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 12263
I20260928 14:42:57.097422 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:42:57.097594 9349 external_mini_cluster.cc:1428] Setting key 1daba5b5ba7f4de34c82a488b08932e7
I20260928 14:42:57.097879 12331 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1eac4563fb094bc19e51e5e39cdf7c5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eac4563fb094bc19e51e5e39cdf7c5b" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37467 } }
I20260928 14:42:57.098009 12331 leader_election.cc:302] T 00000000000000000000000000000000 P 1eac4563fb094bc19e51e5e39cdf7c5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1eac4563fb094bc19e51e5e39cdf7c5b; no voters:
I20260928 14:42:57.098242 12331 leader_election.cc:288] T 00000000000000000000000000000000 P 1eac4563fb094bc19e51e5e39cdf7c5b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:42:57.098273 12336 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1eac4563fb094bc19e51e5e39cdf7c5b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:42:57.098444 12336 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1eac4563fb094bc19e51e5e39cdf7c5b [term 1 LEADER]: Becoming Leader. State: Replica: 1eac4563fb094bc19e51e5e39cdf7c5b, State: Running, Role: LEADER
I20260928 14:42:57.098551 12331 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1eac4563fb094bc19e51e5e39cdf7c5b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:42:57.098588 12336 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1eac4563fb094bc19e51e5e39cdf7c5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eac4563fb094bc19e51e5e39cdf7c5b" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37467 } }
I20260928 14:42:57.098930 12336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1eac4563fb094bc19e51e5e39cdf7c5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1eac4563fb094bc19e51e5e39cdf7c5b. Latest consensus state: current_term: 1 leader_uuid: "1eac4563fb094bc19e51e5e39cdf7c5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eac4563fb094bc19e51e5e39cdf7c5b" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37467 } } }
I20260928 14:42:57.099052 12336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1eac4563fb094bc19e51e5e39cdf7c5b [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:57.098975 12338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1eac4563fb094bc19e51e5e39cdf7c5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1eac4563fb094bc19e51e5e39cdf7c5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eac4563fb094bc19e51e5e39cdf7c5b" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37467 } } }
I20260928 14:42:57.099313 12338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1eac4563fb094bc19e51e5e39cdf7c5b [sys.catalog]: This master's current role is: LEADER
W20260928 14:42:57.100514 12352 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1eac4563fb094bc19e51e5e39cdf7c5b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:42:57.100571 12352 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:42:57.100612 12344 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:42:57.100744 12344 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:42:57.102260 12344 catalog_manager.cc:1347] Generated new cluster ID: 1f02b149f4354ef083713cf679569dd0
I20260928 14:42:57.102313 12344 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 14:42:57 dist-test-slave-pv0w krb5kdc[12250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:42:57.110176 12335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:57.098061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58912 (local address 127.9.33.126:37467)
0928 14:42:57.098270 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:57.098279 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:57.098315 (+ 36us) server_negotiation.cc:407] Connection header received
0928 14:42:57.098479 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:57.098487 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:57.098553 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:57.099542 (+ 989us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:57.099660 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:57.101162 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:57.102219 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:57.102852 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:57.102965 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:57.105783 (+ 2818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:57.105787 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:57.105787 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:57.105800 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:57.108378 (+ 2578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:57.108801 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:57.108804 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:57.108804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:57.108923 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:57.109144 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:57.109144 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:57.109145 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:57.109450 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:57.109607 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:57.109723 (+ 116us) server_negotiation.cc:299] Negotiation successful
0928 14:42:57.109959 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":99,"threads_started":1}
I20260928 14:42:57.111083 9349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:42:57.123888 12344 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:42:57.124622 12344 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:42:57.127880 12335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:57.123508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58924 (local address 127.9.33.126:37467)
0928 14:42:57.123553 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:57.123556 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:57.123755 (+ 199us) server_negotiation.cc:407] Connection header received
0928 14:42:57.123792 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:57.123793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:57.123807 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:57.123884 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:57.124329 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:57.125086 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:57.126111 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:57.126444 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:57.126516 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:57.127298 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:57.127299 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:57.127300 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:42:57.127554 (+ 254us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:57.127577 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:42:57.127587 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:57.127655 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:57.127744 (+ 89us) server_negotiation.cc:299] Negotiation successful
0928 14:42:57.127795 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 14:42:57.132969 12344 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1eac4563fb094bc19e51e5e39cdf7c5b: Generated new TSK 0
I20260928 14:42:57.133179 12344 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:42:57.137085 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 12263
2026-09-28T14:42:57Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1327 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:42:57 dist-test-slave-pv0w krb5kdc[12364](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:42:57 dist-test-slave-pv0w krb5kdc[12364](info): set up 2 sockets
Sep 28 14:42:57 dist-test-slave-pv0w krb5kdc[12364](info): commencing operation
krb5kdc: starting...
W20260928 14:42:58.168414 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:42:58 dist-test-slave-pv0w krb5kdc[12364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:42:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:42:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.33.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
I20260928 14:42:58.290385 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:37625
--webserver_interface=127.9.33.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:34841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:37625
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:58.371354 12377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:58.371591 12377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:58.371629 12377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:58.372779 12377 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:42:58.372850 12377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:58.372874 12377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:58.372895 12377 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:42:58.372924 12377 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:42:58.374366 12377 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:34841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:37625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:37625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:58.374626 12377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:58.374867 12377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:58.379794 12385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:58.379851 12383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:58.379837 12377 server_base.cc:1034] running on GCE node
W20260928 14:42:58.379813 12382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:58.380330 12377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:58.380607 12377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:58.381809 12377 hybrid_clock.cc:648] HybridClock initialized: now 1790606578381798 us; error 39 us; skew 500 ppm
Sep 28 14:42:58 dist-test-slave-pv0w krb5kdc[12364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606578, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:58.388904 12377 init.cc:377] Logged in from keytab as kudu/127.9.33.126@KRBTEST.COM (short username kudu)
I20260928 14:42:58.389456 12377 webserver.cc:460] Webserver started at http://127.9.33.126:37191/ using document root <none> and password file <none>
I20260928 14:42:58.389714 12377 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:58.389791 12377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:58.389962 12377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:58.391173 12377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "9406fdab02f94e03b7d3983812447ec4"
format_stamp: "Formatted at 2026-09-28 14:42:58 on dist-test-slave-pv0w"
server_key: "2df0008491bc576cdd8b80d26cf27d0e"
server_key_iv: "ba415613e1ef4db903347f2d508cc750"
server_key_version: "encryptionkey@0"
I20260928 14:42:58.391585 12377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "9406fdab02f94e03b7d3983812447ec4"
format_stamp: "Formatted at 2026-09-28 14:42:58 on dist-test-slave-pv0w"
server_key: "2df0008491bc576cdd8b80d26cf27d0e"
server_key_iv: "ba415613e1ef4db903347f2d508cc750"
server_key_version: "encryptionkey@0"
I20260928 14:42:58.393083 12377 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:42:58.393921 12392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:58.394105 12377 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:42:58.394232 12377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "9406fdab02f94e03b7d3983812447ec4"
format_stamp: "Formatted at 2026-09-28 14:42:58 on dist-test-slave-pv0w"
server_key: "2df0008491bc576cdd8b80d26cf27d0e"
server_key_iv: "ba415613e1ef4db903347f2d508cc750"
server_key_version: "encryptionkey@0"
I20260928 14:42:58.394317 12377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:58.418464 12377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:58.420989 12377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:58.421164 12377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:58.425426 12377 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:37625
I20260928 14:42:58.425490 12444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:37625 every 8 connection(s)
I20260928 14:42:58.425905 12377 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:42:58.426916 12445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:42:58.429363 12445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9406fdab02f94e03b7d3983812447ec4: Bootstrap starting.
I20260928 14:42:58.429980 12445 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9406fdab02f94e03b7d3983812447ec4: Neither blocks nor log segments found. Creating new log.
I20260928 14:42:58.430282 12445 log.cc:824] T 00000000000000000000000000000000 P 9406fdab02f94e03b7d3983812447ec4: Log is configured to *not* fsync() on all Append() calls
I20260928 14:42:58.431061 12445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9406fdab02f94e03b7d3983812447ec4: No bootstrap required, opened a new log
I20260928 14:42:58.432698 12445 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9406fdab02f94e03b7d3983812447ec4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9406fdab02f94e03b7d3983812447ec4" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37625 } }
I20260928 14:42:58.432847 12445 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9406fdab02f94e03b7d3983812447ec4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:42:58.432870 12445 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9406fdab02f94e03b7d3983812447ec4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9406fdab02f94e03b7d3983812447ec4, State: Initialized, Role: FOLLOWER
I20260928 14:42:58.433046 12445 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9406fdab02f94e03b7d3983812447ec4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9406fdab02f94e03b7d3983812447ec4" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37625 } }
I20260928 14:42:58.433116 12445 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9406fdab02f94e03b7d3983812447ec4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:42:58.433139 12445 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9406fdab02f94e03b7d3983812447ec4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:42:58.433167 12445 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9406fdab02f94e03b7d3983812447ec4 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:42:58.434021 12445 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9406fdab02f94e03b7d3983812447ec4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9406fdab02f94e03b7d3983812447ec4" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37625 } }
I20260928 14:42:58.434116 12445 leader_election.cc:302] T 00000000000000000000000000000000 P 9406fdab02f94e03b7d3983812447ec4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9406fdab02f94e03b7d3983812447ec4; no voters:
I20260928 14:42:58.434334 12445 leader_election.cc:288] T 00000000000000000000000000000000 P 9406fdab02f94e03b7d3983812447ec4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:42:58.434455 12448 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9406fdab02f94e03b7d3983812447ec4 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:42:58.434614 12445 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9406fdab02f94e03b7d3983812447ec4 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:42:58.434619 12448 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9406fdab02f94e03b7d3983812447ec4 [term 1 LEADER]: Becoming Leader. State: Replica: 9406fdab02f94e03b7d3983812447ec4, State: Running, Role: LEADER
I20260928 14:42:58.434753 12448 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9406fdab02f94e03b7d3983812447ec4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9406fdab02f94e03b7d3983812447ec4" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37625 } }
I20260928 14:42:58.435212 12449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9406fdab02f94e03b7d3983812447ec4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9406fdab02f94e03b7d3983812447ec4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9406fdab02f94e03b7d3983812447ec4" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37625 } } }
I20260928 14:42:58.435248 12450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9406fdab02f94e03b7d3983812447ec4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9406fdab02f94e03b7d3983812447ec4. Latest consensus state: current_term: 1 leader_uuid: "9406fdab02f94e03b7d3983812447ec4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9406fdab02f94e03b7d3983812447ec4" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37625 } } }
I20260928 14:42:58.435344 12449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9406fdab02f94e03b7d3983812447ec4 [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:58.435356 12450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9406fdab02f94e03b7d3983812447ec4 [sys.catalog]: This master's current role is: LEADER
I20260928 14:42:58.435600 12457 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:42:58.435672 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 12377
I20260928 14:42:58.435788 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:42:58.436002 9349 external_mini_cluster.cc:1428] Setting key 07da2aaebb967d46f7a1aaf846d85724
I20260928 14:42:58.436187 12457 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:42:58.437789 12457 catalog_manager.cc:1347] Generated new cluster ID: 082055bd096640ffbc71ee7f33136b0b
I20260928 14:42:58.437891 12457 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 14:42:58 dist-test-slave-pv0w krb5kdc[12364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:42:58.446275 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:58.436732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55516 (local address 127.9.33.126:37625)
0928 14:42:58.436973 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:58.436984 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:58.437024 (+ 40us) server_negotiation.cc:407] Connection header received
0928 14:42:58.437123 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:58.437129 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:58.437224 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:58.437367 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:58.438133 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:58.439045 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:58.439885 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:58.440325 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:58.440450 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:58.442801 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:58.442805 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:58.442805 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:58.442837 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:58.444628 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:58.444996 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:58.444999 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:58.445000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:58.445115 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:58.445339 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:58.445340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:58.445340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:58.445607 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:58.445776 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:58.445879 (+ 103us) server_negotiation.cc:299] Negotiation successful
0928 14:42:58.446035 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":82,"threads_started":1}
I20260928 14:42:58.449102 12457 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:42:58.449615 12457 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:42:58.451401 12457 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9406fdab02f94e03b7d3983812447ec4: Generated new TSK 0
I20260928 14:42:58.451593 12457 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.33.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab.
Entry for principal kudu/127.9.33.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab.
I20260928 14:42:58.483186 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.65:0
--local_ip_for_outbound_sockets=127.9.33.65
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:37625
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:34841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:58.591447 12471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:58.591655 12471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:58.591679 12471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:58.592932 12471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:58.592975 12471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:58.593003 12471 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.65
I20260928 14:42:58.594480 12471 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:34841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:37625
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12471
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.65
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:58.594749 12471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:58.595031 12471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:58.600594 12477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:58.600667 12471 server_base.cc:1034] running on GCE node
W20260928 14:42:58.600731 12476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:58.600603 12479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:58.601089 12471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:58.601351 12471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:58.602504 12471 hybrid_clock.cc:648] HybridClock initialized: now 1790606578602477 us; error 42 us; skew 500 ppm
Sep 28 14:42:58 dist-test-slave-pv0w krb5kdc[12364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606578, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:58.605237 12471 init.cc:377] Logged in from keytab as kudu/127.9.33.65@KRBTEST.COM (short username kudu)
I20260928 14:42:58.605724 12471 webserver.cc:460] Webserver started at http://127.9.33.65:37109/ using document root <none> and password file <none>
I20260928 14:42:58.605926 12471 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:58.605969 12471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:58.606058 12471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:58.607218 12471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-0/data/instance:
uuid: "a33b1fc493b64f589b1bcdca43f33ea6"
format_stamp: "Formatted at 2026-09-28 14:42:58 on dist-test-slave-pv0w"
server_key: "314c8d46f324217bdc2618fd3a798eaf"
server_key_iv: "861fdbea3c5e0371fb9a214c7cb76356"
server_key_version: "encryptionkey@0"
I20260928 14:42:58.607599 12471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance:
uuid: "a33b1fc493b64f589b1bcdca43f33ea6"
format_stamp: "Formatted at 2026-09-28 14:42:58 on dist-test-slave-pv0w"
server_key: "314c8d46f324217bdc2618fd3a798eaf"
server_key_iv: "861fdbea3c5e0371fb9a214c7cb76356"
server_key_version: "encryptionkey@0"
I20260928 14:42:58.609095 12471 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:42:58.609838 12486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:58.610045 12471 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:42:58.610138 12471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-0/wal
uuid: "a33b1fc493b64f589b1bcdca43f33ea6"
format_stamp: "Formatted at 2026-09-28 14:42:58 on dist-test-slave-pv0w"
server_key: "314c8d46f324217bdc2618fd3a798eaf"
server_key_iv: "861fdbea3c5e0371fb9a214c7cb76356"
server_key_version: "encryptionkey@0"
I20260928 14:42:58.610214 12471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:58.621096 12471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:58.623606 12471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:58.623845 12471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:58.624117 12471 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:42:58.624464 12471 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:42:58.624522 12471 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:58.624560 12471 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:42:58.624589 12471 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:58.630970 12471 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.65:33435
I20260928 14:42:58.631116 12599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.65:33435 every 8 connection(s)
I20260928 14:42:58.631421 12471 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
Sep 28 14:42:58 dist-test-slave-pv0w krb5kdc[12364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606578, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:42:58.639245 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 12471
I20260928 14:42:58.639385 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance
I20260928 14:42:58.639658 9349 external_mini_cluster.cc:1428] Setting key 1b66a76cd90e0b51f60c32d71053a485
I20260928 14:42:58.642544 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:58.632430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:40927 (local address 127.9.33.126:37625)
0928 14:42:58.632527 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:58.632530 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:58.632776 (+ 246us) server_negotiation.cc:407] Connection header received
0928 14:42:58.633409 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:58.633410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:58.633423 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:58.633479 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:58.634592 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:58.635201 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:58.636381 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:58.636622 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:58.636713 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:58.639263 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:58.639265 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:58.639266 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:58.639275 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:58.640978 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:58.641469 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:58.641471 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:58.641472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:58.641534 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:58.641811 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:58.641812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:58.641813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:58.641952 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:58.642027 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:58.642350 (+ 323us) server_negotiation.cc:299] Negotiation successful
0928 14:42:58.642449 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 14:42:58.642767 12602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:58.632305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37625 (local address 127.9.33.65:40927)
0928 14:42:58.632546 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:58.632656 (+ 110us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:58.633278 (+ 622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:58.633556 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:58.633560 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:58.634234 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:58.634469 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:58.634475 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:58.635311 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:58.635312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:58.636249 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:58.636251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:58.636727 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:58.636728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:58.636867 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:58.637456 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:58.637466 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:58.639144 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:58.641087 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:58.641098 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:58.641101 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:58.641364 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:58.641610 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:58.641611 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:58.641611 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:58.641740 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:58.642029 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:58.642038 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:58.642203 (+ 165us) client_negotiation.cc:769] Sending connection context
0928 14:42:58.642316 (+ 113us) client_negotiation.cc:240] Negotiation successful
0928 14:42:58.642509 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":110,"threads_started":1}
I20260928 14:42:58.643474 12600 heartbeater.cc:347] Connected to a master server at 127.9.33.126:37625
I20260928 14:42:58.643613 12600 heartbeater.cc:464] Registering TS with master...
I20260928 14:42:58.643873 12600 heartbeater.cc:511] Master 127.9.33.126:37625 requested a full tablet report, sending...
I20260928 14:42:58.644583 12409 ts_manager.cc:194] Registered new tserver with Master: a33b1fc493b64f589b1bcdca43f33ea6 (127.9.33.65:33435)
I20260928 14:42:58.645685 12409 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.65@KRBTEST.COM'} at 127.9.33.65:40927
WARNING: no policy specified for kudu/127.9.33.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab.
Entry for principal kudu/127.9.33.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab.
I20260928 14:42:58.667490 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.66:0
--local_ip_for_outbound_sockets=127.9.33.66
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:37625
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:34841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:58.751085 12605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:58.751379 12605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:58.751426 12605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:58.752838 12605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:58.752918 12605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:58.752965 12605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.66
I20260928 14:42:58.754580 12605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:34841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:37625
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.66
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:58.754930 12605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:58.755208 12605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:58.760499 12610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:58.760499 12613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:58.760524 12611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:58.760869 12605 server_base.cc:1034] running on GCE node
I20260928 14:42:58.761094 12605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:58.761358 12605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:58.762502 12605 hybrid_clock.cc:648] HybridClock initialized: now 1790606578762483 us; error 36 us; skew 500 ppm
Sep 28 14:42:58 dist-test-slave-pv0w krb5kdc[12364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606578, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:58.765097 12605 init.cc:377] Logged in from keytab as kudu/127.9.33.66@KRBTEST.COM (short username kudu)
I20260928 14:42:58.765599 12605 webserver.cc:460] Webserver started at http://127.9.33.66:42135/ using document root <none> and password file <none>
I20260928 14:42:58.765844 12605 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:58.765914 12605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:58.766064 12605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:58.767213 12605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-1/data/instance:
uuid: "e0849517919e49a68508502141b425d1"
format_stamp: "Formatted at 2026-09-28 14:42:58 on dist-test-slave-pv0w"
server_key: "9d13d4e36672f40cdd9d3c0546afa25b"
server_key_iv: "f8af7274da25cfab0c8d85e1c1f8ff98"
server_key_version: "encryptionkey@0"
I20260928 14:42:58.767591 12605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance:
uuid: "e0849517919e49a68508502141b425d1"
format_stamp: "Formatted at 2026-09-28 14:42:58 on dist-test-slave-pv0w"
server_key: "9d13d4e36672f40cdd9d3c0546afa25b"
server_key_iv: "f8af7274da25cfab0c8d85e1c1f8ff98"
server_key_version: "encryptionkey@0"
I20260928 14:42:58.769145 12605 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:42:58.770001 12620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:58.770195 12605 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260928 14:42:58.770303 12605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-1/wal
uuid: "e0849517919e49a68508502141b425d1"
format_stamp: "Formatted at 2026-09-28 14:42:58 on dist-test-slave-pv0w"
server_key: "9d13d4e36672f40cdd9d3c0546afa25b"
server_key_iv: "f8af7274da25cfab0c8d85e1c1f8ff98"
server_key_version: "encryptionkey@0"
I20260928 14:42:58.770382 12605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:58.785444 12605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:58.788709 12605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:58.788890 12605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:58.789193 12605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:42:58.789664 12605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:42:58.789716 12605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:58.789753 12605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:42:58.789767 12605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:58.795706 12605 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.66:43373
I20260928 14:42:58.795830 12733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.66:43373 every 8 connection(s)
I20260928 14:42:58.796195 12605 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
Sep 28 14:42:58 dist-test-slave-pv0w krb5kdc[12364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606578, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:42:58.803112 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 12605
I20260928 14:42:58.803228 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance
I20260928 14:42:58.803458 9349 external_mini_cluster.cc:1428] Setting key b739fec94c58de26f7b7162f6c858871
I20260928 14:42:58.806634 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:58.797267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:34709 (local address 127.9.33.126:37625)
0928 14:42:58.797402 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:58.797404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:58.797607 (+ 203us) server_negotiation.cc:407] Connection header received
0928 14:42:58.798186 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:58.798187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:58.798198 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:58.798281 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:58.799314 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:58.799910 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:58.800922 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:58.801114 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:58.801172 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:58.803437 (+ 2265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:58.803438 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:58.803439 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:58.803446 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:58.805121 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:58.805523 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:58.805525 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:58.805525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:58.805590 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:58.805914 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:58.805919 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:58.805920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:58.806080 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:58.806135 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:58.806503 (+ 368us) server_negotiation.cc:299] Negotiation successful
0928 14:42:58.806569 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260928 14:42:58.806921 12736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:58.797217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37625 (local address 127.9.33.66:34709)
0928 14:42:58.797443 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:58.797494 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:58.798067 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:58.798413 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:58.798417 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:58.799019 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:58.799218 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:58.799223 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:58.799986 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:58.799987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:58.800814 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:58.800817 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:58.801191 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:58.801192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:58.801308 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:58.801875 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:58.801884 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:58.803323 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:58.805208 (+ 1885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:58.805215 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:58.805216 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:58.805449 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:58.805656 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:58.805656 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:58.805656 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:58.805833 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:58.806150 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:58.806156 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:58.806358 (+ 202us) client_negotiation.cc:769] Sending connection context
0928 14:42:58.806485 (+ 127us) client_negotiation.cc:240] Negotiation successful
0928 14:42:58.806664 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":143,"threads_started":1}
I20260928 14:42:58.807520 12734 heartbeater.cc:347] Connected to a master server at 127.9.33.126:37625
I20260928 14:42:58.807633 12734 heartbeater.cc:464] Registering TS with master...
I20260928 14:42:58.807861 12734 heartbeater.cc:511] Master 127.9.33.126:37625 requested a full tablet report, sending...
I20260928 14:42:58.808321 12409 ts_manager.cc:194] Registered new tserver with Master: e0849517919e49a68508502141b425d1 (127.9.33.66:43373)
I20260928 14:42:58.808759 12409 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.66@KRBTEST.COM'} at 127.9.33.66:34709
WARNING: no policy specified for kudu/127.9.33.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab.
Entry for principal kudu/127.9.33.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab.
I20260928 14:42:58.831440 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.67:0
--local_ip_for_outbound_sockets=127.9.33.67
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:37625
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:34841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:42:58.912787 12739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:42:58.913116 12739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:42:58.913158 12739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:42:58.914490 12739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:42:58.914567 12739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:42:58.914618 12739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.67
I20260928 14:42:58.916296 12739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:34841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:37625
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.67
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:42:58.916596 12739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:42:58.916850 12739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:42:58.922034 12747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:42:58.922044 12745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:58.922353 12739 server_base.cc:1034] running on GCE node
W20260928 14:42:58.922034 12744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:42:58.922770 12739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:42:58.922993 12739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:42:58.924158 12739 hybrid_clock.cc:648] HybridClock initialized: now 1790606578924149 us; error 42 us; skew 500 ppm
Sep 28 14:42:58 dist-test-slave-pv0w krb5kdc[12364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606578, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:42:58.926656 12739 init.cc:377] Logged in from keytab as kudu/127.9.33.67@KRBTEST.COM (short username kudu)
I20260928 14:42:58.927158 12739 webserver.cc:460] Webserver started at http://127.9.33.67:34525/ using document root <none> and password file <none>
I20260928 14:42:58.927407 12739 fs_manager.cc:359] Metadata directory not provided
I20260928 14:42:58.927476 12739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:42:58.927637 12739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:42:58.928781 12739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-2/data/instance:
uuid: "88d23b4def664cf2a6a7d4bdfd589c6b"
format_stamp: "Formatted at 2026-09-28 14:42:58 on dist-test-slave-pv0w"
server_key: "a9e863b408d0e29b78af3d6345f07614"
server_key_iv: "3f7f67a794199e74f0d6a777a00d99df"
server_key_version: "encryptionkey@0"
I20260928 14:42:58.929172 12739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance:
uuid: "88d23b4def664cf2a6a7d4bdfd589c6b"
format_stamp: "Formatted at 2026-09-28 14:42:58 on dist-test-slave-pv0w"
server_key: "a9e863b408d0e29b78af3d6345f07614"
server_key_iv: "3f7f67a794199e74f0d6a777a00d99df"
server_key_version: "encryptionkey@0"
I20260928 14:42:58.930612 12739 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:42:58.931430 12754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:58.931622 12739 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:42:58.931766 12739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-2/wal
uuid: "88d23b4def664cf2a6a7d4bdfd589c6b"
format_stamp: "Formatted at 2026-09-28 14:42:58 on dist-test-slave-pv0w"
server_key: "a9e863b408d0e29b78af3d6345f07614"
server_key_iv: "3f7f67a794199e74f0d6a777a00d99df"
server_key_version: "encryptionkey@0"
I20260928 14:42:58.931849 12739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:42:58.942634 12739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:42:58.944767 12739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:42:58.944952 12739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:42:58.945196 12739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:42:58.945523 12739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:42:58.945580 12739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:58.945626 12739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:42:58.945650 12739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:42:58.951838 12739 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.67:43733
I20260928 14:42:58.951905 12867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.67:43733 every 8 connection(s)
I20260928 14:42:58.952279 12739 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
I20260928 14:42:58.956451 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 12739
I20260928 14:42:58.956537 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance
I20260928 14:42:58.956774 9349 external_mini_cluster.cc:1428] Setting key 83c2499e22fac8b1528517496fda5c3e
Sep 28 14:42:58 dist-test-slave-pv0w krb5kdc[12364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606578, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:42:58.961874 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:58.953035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:49097 (local address 127.9.33.126:37625)
0928 14:42:58.953079 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:58.953082 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:58.953316 (+ 234us) server_negotiation.cc:407] Connection header received
0928 14:42:58.953798 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:58.953799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:58.953810 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:58.953859 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:58.954844 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:58.955346 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:58.956462 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:58.956729 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:58.956819 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:58.959099 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:58.959100 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:58.959100 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:58.959108 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:58.960355 (+ 1247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:58.960869 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:58.960870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:58.960871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:58.960943 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:58.961209 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:58.961210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:58.961210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:58.961333 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:58.961409 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:58.961697 (+ 288us) server_negotiation.cc:299] Negotiation successful
0928 14:42:58.961780 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 14:42:58.962080 12870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:58.953039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37625 (local address 127.9.33.67:49097)
0928 14:42:58.953190 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0928 14:42:58.953229 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 14:42:58.953714 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:42:58.953934 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:42:58.953937 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:42:58.954540 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:42:58.954757 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:58.954763 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:58.955431 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:58.955432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:58.956354 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:42:58.956356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:42:58.956834 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:42:58.956835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:42:58.957011 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:58.957618 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:42:58.957626 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:42:58.958984 (+ 1358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:42:58.960467 (+ 1483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:58.960475 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:58.960478 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:58.960790 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:58.961012 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:42:58.961013 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:42:58.961013 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:42:58.961133 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:42:58.961406 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:42:58.961414 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:42:58.961561 (+ 147us) client_negotiation.cc:769] Sending connection context
0928 14:42:58.961674 (+ 113us) client_negotiation.cc:240] Negotiation successful
0928 14:42:58.961839 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":88,"threads_started":1}
I20260928 14:42:58.962574 12868 heartbeater.cc:347] Connected to a master server at 127.9.33.126:37625
I20260928 14:42:58.962656 12868 heartbeater.cc:464] Registering TS with master...
I20260928 14:42:58.962800 12868 heartbeater.cc:511] Master 127.9.33.126:37625 requested a full tablet report, sending...
I20260928 14:42:58.963177 12409 ts_manager.cc:194] Registered new tserver with Master: 88d23b4def664cf2a6a7d4bdfd589c6b (127.9.33.67:43733)
I20260928 14:42:58.963574 12409 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.67@KRBTEST.COM'} at 127.9.33.67:49097
I20260928 14:42:58.968427 9349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 28 14:42:58 dist-test-slave-pv0w krb5kdc[12364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606578, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 28 14:42:58 dist-test-slave-pv0w krb5kdc[12364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606578, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:42:58.990993 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:42:58.983870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55532 (local address 127.9.33.126:37625)
0928 14:42:58.983971 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0928 14:42:58.983974 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:42:58.984078 (+ 104us) server_negotiation.cc:407] Connection header received
0928 14:42:58.984168 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:42:58.984169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:42:58.984183 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:42:58.984252 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:42:58.984856 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:58.985365 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:58.986128 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:42:58.986325 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:42:58.986409 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:42:58.988723 (+ 2314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:42:58.988725 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:42:58.988725 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:42:58.988732 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:42:58.989986 (+ 1254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:58.990295 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:58.990295 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:58.990296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:58.990348 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:42:58.990528 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:42:58.990528 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:42:58.990529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:42:58.990670 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:42:58.990744 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 14:42:58.990847 (+ 103us) server_negotiation.cc:299] Negotiation successful
0928 14:42:58.990925 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260928 14:42:58.991267 12409 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:55532
I20260928 14:42:58.991820 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 12471
I20260928 14:42:58.996574 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 12605
I20260928 14:42:59.001317 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 12739
I20260928 14:42:59.005589 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 12377
2026-09-28T14:42:59Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1870 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:42:59 dist-test-slave-pv0w krb5kdc[12880](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:42:59 dist-test-slave-pv0w krb5kdc[12880](info): set up 2 sockets
Sep 28 14:42:59 dist-test-slave-pv0w krb5kdc[12880](info): commencing operation
krb5kdc: starting...
W20260928 14:43:00.118297 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260928 14:43:00.347549 9349 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.229s user 0.002s sys 0.000s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260928 14:43:00.490926 9349 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.143s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260928 14:43:00.808287 9349 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.317s user 0.000s sys 0.002s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/krb5kdc/test-user.keytab.
W20260928 14:43:01.438710 9349 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.630s user 0.000s sys 0.001s
Sep 28 14:43:01 dist-test-slave-pv0w krb5kdc[12880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:43:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:43:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.33.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
W20260928 14:43:02.834906 9349 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.9.33.126: real 1.374s user 0.000s sys 0.003s
I20260928 14:43:02.835389 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:43245
--webserver_interface=127.9.33.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:45095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:43245
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:43:02.921033 12893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:02.921283 12893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:02.921319 12893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:02.922494 12893 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:43:02.922564 12893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:02.922585 12893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:02.922605 12893 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:43:02.922637 12893 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:43:02.924185 12893 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:45095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:43245
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:43245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:02.924472 12893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:02.924703 12893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:02.929880 12901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:02.929858 12898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:02.930099 12899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:02.930238 12893 server_base.cc:1034] running on GCE node
I20260928 14:43:02.930510 12893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:02.930862 12893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:02.932060 12893 hybrid_clock.cc:648] HybridClock initialized: now 1790606582932040 us; error 51 us; skew 500 ppm
Sep 28 14:43:02 dist-test-slave-pv0w krb5kdc[12880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606582, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:02.934890 12893 init.cc:377] Logged in from keytab as kudu/127.9.33.126@KRBTEST.COM (short username kudu)
I20260928 14:43:02.935369 12893 webserver.cc:460] Webserver started at http://127.9.33.126:36439/ using document root <none> and password file <none>
I20260928 14:43:02.935619 12893 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:02.935693 12893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:02.935910 12893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:02.937062 12893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "f895375ae68f46e7bee0307cf2edf05d"
format_stamp: "Formatted at 2026-09-28 14:43:02 on dist-test-slave-pv0w"
server_key: "385581b9f3e49581f32c60d42d458e00"
server_key_iv: "c616586391de038ffdf4e9ba43585bba"
server_key_version: "encryptionkey@0"
I20260928 14:43:02.937453 12893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "f895375ae68f46e7bee0307cf2edf05d"
format_stamp: "Formatted at 2026-09-28 14:43:02 on dist-test-slave-pv0w"
server_key: "385581b9f3e49581f32c60d42d458e00"
server_key_iv: "c616586391de038ffdf4e9ba43585bba"
server_key_version: "encryptionkey@0"
I20260928 14:43:02.939079 12893 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:43:02.940084 12908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:02.940331 12893 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:02.940476 12893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "f895375ae68f46e7bee0307cf2edf05d"
format_stamp: "Formatted at 2026-09-28 14:43:02 on dist-test-slave-pv0w"
server_key: "385581b9f3e49581f32c60d42d458e00"
server_key_iv: "c616586391de038ffdf4e9ba43585bba"
server_key_version: "encryptionkey@0"
I20260928 14:43:02.940567 12893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:02.951694 12893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:02.954362 12893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:02.954546 12893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:02.958853 12893 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:43245
I20260928 14:43:02.958976 12960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:43245 every 8 connection(s)
I20260928 14:43:02.959453 12893 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:43:02.960227 12961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:02.960871 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 12893
I20260928 14:43:02.960988 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:43:02.961171 9349 external_mini_cluster.cc:1428] Setting key 127fab93d9cebfabd9064afe076fa42a
I20260928 14:43:02.963573 12961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f895375ae68f46e7bee0307cf2edf05d: Bootstrap starting.
I20260928 14:43:02.964248 12961 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f895375ae68f46e7bee0307cf2edf05d: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:02.964555 12961 log.cc:824] T 00000000000000000000000000000000 P f895375ae68f46e7bee0307cf2edf05d: Log is configured to *not* fsync() on all Append() calls
I20260928 14:43:02.965265 12961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f895375ae68f46e7bee0307cf2edf05d: No bootstrap required, opened a new log
I20260928 14:43:02.967262 12961 raft_consensus.cc:348] T 00000000000000000000000000000000 P f895375ae68f46e7bee0307cf2edf05d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f895375ae68f46e7bee0307cf2edf05d" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 43245 } }
I20260928 14:43:02.967443 12961 raft_consensus.cc:374] T 00000000000000000000000000000000 P f895375ae68f46e7bee0307cf2edf05d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:02.967483 12961 raft_consensus.cc:729] T 00000000000000000000000000000000 P f895375ae68f46e7bee0307cf2edf05d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f895375ae68f46e7bee0307cf2edf05d, State: Initialized, Role: FOLLOWER
I20260928 14:43:02.967638 12961 consensus_queue.cc:260] T 00000000000000000000000000000000 P f895375ae68f46e7bee0307cf2edf05d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f895375ae68f46e7bee0307cf2edf05d" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 43245 } }
I20260928 14:43:02.967752 12961 raft_consensus.cc:388] T 00000000000000000000000000000000 P f895375ae68f46e7bee0307cf2edf05d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:43:02.967794 12961 raft_consensus.cc:482] T 00000000000000000000000000000000 P f895375ae68f46e7bee0307cf2edf05d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:43:02.967852 12961 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f895375ae68f46e7bee0307cf2edf05d [term 0 FOLLOWER]: Advancing to term 1
Sep 28 14:43:02 dist-test-slave-pv0w krb5kdc[12880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:43:02.972671 12964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:02.961816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46716 (local address 127.9.33.126:43245)
0928 14:43:02.961995 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:02.962004 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:02.962030 (+ 26us) server_negotiation.cc:407] Connection header received
0928 14:43:02.962452 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:02.962459 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:02.962507 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:02.962628 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:02.963468 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:02.964650 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:02.965471 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:02.965902 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:02.966028 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:02.968696 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:02.968700 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:02.968700 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:02.968711 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:02.970376 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:02.970752 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:02.970756 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:02.970756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:02.970909 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:02.971193 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:02.971195 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:02.971195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:02.972111 (+ 916us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:02.972234 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:02.972363 (+ 129us) server_negotiation.cc:299] Negotiation successful
0928 14:43:02.972490 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":87,"threads_started":1}
I20260928 14:43:03.030707 12961 raft_consensus.cc:504] T 00000000000000000000000000000000 P f895375ae68f46e7bee0307cf2edf05d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f895375ae68f46e7bee0307cf2edf05d" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 43245 } }
I20260928 14:43:03.030921 12961 leader_election.cc:302] T 00000000000000000000000000000000 P f895375ae68f46e7bee0307cf2edf05d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f895375ae68f46e7bee0307cf2edf05d; no voters:
I20260928 14:43:03.031200 12961 leader_election.cc:288] T 00000000000000000000000000000000 P f895375ae68f46e7bee0307cf2edf05d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:43:03.031299 12966 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f895375ae68f46e7bee0307cf2edf05d [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:43:03.031554 12966 raft_consensus.cc:686] T 00000000000000000000000000000000 P f895375ae68f46e7bee0307cf2edf05d [term 1 LEADER]: Becoming Leader. State: Replica: f895375ae68f46e7bee0307cf2edf05d, State: Running, Role: LEADER
I20260928 14:43:03.031535 12961 sys_catalog.cc:564] T 00000000000000000000000000000000 P f895375ae68f46e7bee0307cf2edf05d [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:43:03.031793 12966 consensus_queue.cc:237] T 00000000000000000000000000000000 P f895375ae68f46e7bee0307cf2edf05d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f895375ae68f46e7bee0307cf2edf05d" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 43245 } }
I20260928 14:43:03.032579 12967 sys_catalog.cc:455] T 00000000000000000000000000000000 P f895375ae68f46e7bee0307cf2edf05d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f895375ae68f46e7bee0307cf2edf05d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f895375ae68f46e7bee0307cf2edf05d" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 43245 } } }
I20260928 14:43:03.032680 12967 sys_catalog.cc:458] T 00000000000000000000000000000000 P f895375ae68f46e7bee0307cf2edf05d [sys.catalog]: This master's current role is: LEADER
I20260928 14:43:03.033015 12968 sys_catalog.cc:455] T 00000000000000000000000000000000 P f895375ae68f46e7bee0307cf2edf05d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f895375ae68f46e7bee0307cf2edf05d. Latest consensus state: current_term: 1 leader_uuid: "f895375ae68f46e7bee0307cf2edf05d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f895375ae68f46e7bee0307cf2edf05d" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 43245 } } }
I20260928 14:43:03.033190 12968 sys_catalog.cc:458] T 00000000000000000000000000000000 P f895375ae68f46e7bee0307cf2edf05d [sys.catalog]: This master's current role is: LEADER
W20260928 14:43:03.033977 12982 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f895375ae68f46e7bee0307cf2edf05d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:43:03.034051 12982 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:43:03.034101 12975 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:43:03.034296 12975 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:43:03.036733 12975 catalog_manager.cc:1347] Generated new cluster ID: d4ba12d300f64faeb896685e98a8df99
I20260928 14:43:03.036823 12975 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:43:03.046310 12975 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:43:03.046947 12975 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:43:03.053767 12975 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f895375ae68f46e7bee0307cf2edf05d: Generated new TSK 0
I20260928 14:43:03.054042 12975 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.33.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab.
Entry for principal kudu/127.9.33.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab.
W20260928 14:43:03.603742 9349 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.9.33.65: real 0.542s user 0.000s sys 0.003s
I20260928 14:43:03.604218 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.65:0
--local_ip_for_outbound_sockets=127.9.33.65
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:43245
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:45095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:43:03.688128 12987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:03.688393 12987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:03.688432 12987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:03.689647 12987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:03.689726 12987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:03.689774 12987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.65
I20260928 14:43:03.691365 12987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:45095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:43245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.65
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:03.691648 12987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:03.691932 12987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:03.697418 12992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:03.697412 12993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:03.697412 12995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:03.697738 12987 server_base.cc:1034] running on GCE node
I20260928 14:43:03.698089 12987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:03.698344 12987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:03.699566 12987 hybrid_clock.cc:648] HybridClock initialized: now 1790606583699538 us; error 41 us; skew 500 ppm
Sep 28 14:43:03 dist-test-slave-pv0w krb5kdc[12880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606583, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:03.710139 12987 init.cc:377] Logged in from keytab as kudu/127.9.33.65@KRBTEST.COM (short username kudu)
I20260928 14:43:03.710713 12987 webserver.cc:460] Webserver started at http://127.9.33.65:45705/ using document root <none> and password file <none>
I20260928 14:43:03.710989 12987 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:03.711068 12987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:03.711225 12987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:03.712421 12987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-0/data/instance:
uuid: "b8af7c1146bb42798480504c69104ebc"
format_stamp: "Formatted at 2026-09-28 14:43:03 on dist-test-slave-pv0w"
server_key: "99b8e40d147898d516322394cd235caf"
server_key_iv: "740702a10bba1933807bf64b00488010"
server_key_version: "encryptionkey@0"
I20260928 14:43:03.712809 12987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance:
uuid: "b8af7c1146bb42798480504c69104ebc"
format_stamp: "Formatted at 2026-09-28 14:43:03 on dist-test-slave-pv0w"
server_key: "99b8e40d147898d516322394cd235caf"
server_key_iv: "740702a10bba1933807bf64b00488010"
server_key_version: "encryptionkey@0"
I20260928 14:43:03.714254 12987 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:43:03.715098 13002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:03.715303 12987 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:03.715425 12987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-0/wal
uuid: "b8af7c1146bb42798480504c69104ebc"
format_stamp: "Formatted at 2026-09-28 14:43:03 on dist-test-slave-pv0w"
server_key: "99b8e40d147898d516322394cd235caf"
server_key_iv: "740702a10bba1933807bf64b00488010"
server_key_version: "encryptionkey@0"
I20260928 14:43:03.715505 12987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:03.743147 12987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:03.745424 12987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:03.745603 12987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:03.745877 12987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:43:03.746217 12987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:43:03.746282 12987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:03.746325 12987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:43:03.746356 12987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:03.752731 12987 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.65:43413
I20260928 14:43:03.752772 13115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.65:43413 every 8 connection(s)
I20260928 14:43:03.753118 12987 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
Sep 28 14:43:03 dist-test-slave-pv0w krb5kdc[12880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606583, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:43:03.759989 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 12987
I20260928 14:43:03.760107 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance
I20260928 14:43:03.760320 9349 external_mini_cluster.cc:1428] Setting key b392ce273e52b2ff3c1809bee7097685
I20260928 14:43:03.763803 13118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:03.753955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:50157 (local address 127.9.33.126:43245)
0928 14:43:03.754095 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:03.754098 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:03.754291 (+ 193us) server_negotiation.cc:407] Connection header received
0928 14:43:03.755325 (+ 1034us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:03.755326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:03.755339 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:03.755407 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:03.756139 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:03.756670 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:03.757716 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:03.757912 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:03.757965 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:03.760234 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:03.760236 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:03.760236 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:03.760244 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:03.762228 (+ 1984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:03.762686 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:03.762689 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:03.762689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:03.762764 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:03.763042 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:03.763043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:03.763044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:03.763201 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:03.763286 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:03.763582 (+ 296us) server_negotiation.cc:299] Negotiation successful
0928 14:43:03.763674 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":63,"threads_started":1}
I20260928 14:43:03.764016 13119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:03.753935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:43245 (local address 127.9.33.65:50157)
0928 14:43:03.754132 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:03.754175 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:03.755176 (+ 1001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:03.755490 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:03.755493 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:03.755846 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:03.756058 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:03.756063 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:03.756780 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:03.756781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:03.757623 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:03.757625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:03.757992 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:03.757993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:03.758103 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:03.758610 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:03.758619 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:03.760125 (+ 1506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:03.762332 (+ 2207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:03.762338 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:03.762340 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:03.762567 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:03.762838 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:03.762839 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:03.762839 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:03.762969 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:03.763275 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:03.763282 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:03.763442 (+ 160us) client_negotiation.cc:769] Sending connection context
0928 14:43:03.763560 (+ 118us) client_negotiation.cc:240] Negotiation successful
0928 14:43:03.763731 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":107,"threads_started":1}
I20260928 14:43:03.764509 13116 heartbeater.cc:347] Connected to a master server at 127.9.33.126:43245
I20260928 14:43:03.764611 13116 heartbeater.cc:464] Registering TS with master...
I20260928 14:43:03.764801 13116 heartbeater.cc:511] Master 127.9.33.126:43245 requested a full tablet report, sending...
I20260928 14:43:03.765426 12925 ts_manager.cc:194] Registered new tserver with Master: b8af7c1146bb42798480504c69104ebc (127.9.33.65:43413)
I20260928 14:43:03.766390 12925 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.65@KRBTEST.COM'} at 127.9.33.65:50157
WARNING: no policy specified for kudu/127.9.33.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab.
Entry for principal kudu/127.9.33.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab.
I20260928 14:43:03.786875 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.66:0
--local_ip_for_outbound_sockets=127.9.33.66
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:43245
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:45095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:43:03.871104 13122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:03.871335 13122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:03.871361 13122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:03.872630 13122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:03.872684 13122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:03.872713 13122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.66
I20260928 14:43:03.874425 13122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:45095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:43245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.66
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:03.874694 13122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:03.874981 13122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:03.880232 13128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:03.880259 13130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:03.880266 13127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:03.880754 13122 server_base.cc:1034] running on GCE node
I20260928 14:43:03.880999 13122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:03.881258 13122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:03.882450 13122 hybrid_clock.cc:648] HybridClock initialized: now 1790606583882450 us; error 45 us; skew 500 ppm
Sep 28 14:43:03 dist-test-slave-pv0w krb5kdc[12880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606583, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:03.885221 13122 init.cc:377] Logged in from keytab as kudu/127.9.33.66@KRBTEST.COM (short username kudu)
I20260928 14:43:03.885733 13122 webserver.cc:460] Webserver started at http://127.9.33.66:45537/ using document root <none> and password file <none>
I20260928 14:43:03.885980 13122 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:03.886051 13122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:03.886204 13122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:03.887873 13122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-1/data/instance:
uuid: "50956d05116c423cbe5439996042698d"
format_stamp: "Formatted at 2026-09-28 14:43:03 on dist-test-slave-pv0w"
server_key: "2c2450ebaaec1855f8d62012b1717a34"
server_key_iv: "d94596935b2ac3d7ca77cc5345ed05ac"
server_key_version: "encryptionkey@0"
I20260928 14:43:03.888257 13122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance:
uuid: "50956d05116c423cbe5439996042698d"
format_stamp: "Formatted at 2026-09-28 14:43:03 on dist-test-slave-pv0w"
server_key: "2c2450ebaaec1855f8d62012b1717a34"
server_key_iv: "d94596935b2ac3d7ca77cc5345ed05ac"
server_key_version: "encryptionkey@0"
I20260928 14:43:03.889748 13122 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:03.890558 13137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:03.890815 13122 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:03.890930 13122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-1/wal
uuid: "50956d05116c423cbe5439996042698d"
format_stamp: "Formatted at 2026-09-28 14:43:03 on dist-test-slave-pv0w"
server_key: "2c2450ebaaec1855f8d62012b1717a34"
server_key_iv: "d94596935b2ac3d7ca77cc5345ed05ac"
server_key_version: "encryptionkey@0"
I20260928 14:43:03.891019 13122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:03.907707 13122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:03.909984 13122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:03.910181 13122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:03.910447 13122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:43:03.910791 13122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:43:03.910848 13122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:03.910892 13122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:43:03.910974 13122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:03.917054 13122 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.66:40287
I20260928 14:43:03.917160 13250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.66:40287 every 8 connection(s)
I20260928 14:43:03.917476 13122 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
I20260928 14:43:03.922257 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 13122
I20260928 14:43:03.922353 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance
I20260928 14:43:03.922549 9349 external_mini_cluster.cc:1428] Setting key 060e7ac180c6327fd2fc0a389b5b501e
Sep 28 14:43:03 dist-test-slave-pv0w krb5kdc[12880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606583, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:43:03.928400 13118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:03.918292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:60967 (local address 127.9.33.126:43245)
0928 14:43:03.918365 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:03.918366 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:03.918610 (+ 244us) server_negotiation.cc:407] Connection header received
0928 14:43:03.919607 (+ 997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:03.919608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:03.919620 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:03.919690 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:03.920412 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:03.920925 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:03.922115 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:03.922355 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:03.922431 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:03.924978 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:03.924980 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:03.924980 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:03.924987 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:03.926810 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:03.927262 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:03.927264 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:03.927265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:03.927333 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:03.927606 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:03.927607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:03.927608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:03.927792 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:03.927880 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:03.928248 (+ 368us) server_negotiation.cc:299] Negotiation successful
0928 14:43:03.928332 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260928 14:43:03.928738 13253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:03.918276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:43245 (local address 127.9.33.66:60967)
0928 14:43:03.918441 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:03.918491 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:03.919506 (+ 1015us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:03.919818 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:03.919829 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:03.920113 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:03.920320 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:03.920325 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:03.921013 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:03.921014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:03.921995 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:03.921998 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:03.922504 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:03.922505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:03.922658 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:03.923228 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:03.923237 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:03.924874 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:03.926899 (+ 2025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:03.926908 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:03.926910 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:03.927169 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:03.927405 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:03.927406 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:03.927407 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:03.927525 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:03.927868 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:03.927876 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:03.928077 (+ 201us) client_negotiation.cc:769] Sending connection context
0928 14:43:03.928214 (+ 137us) client_negotiation.cc:240] Negotiation successful
0928 14:43:03.928430 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":78,"threads_started":1}
I20260928 14:43:03.929288 13251 heartbeater.cc:347] Connected to a master server at 127.9.33.126:43245
I20260928 14:43:03.929394 13251 heartbeater.cc:464] Registering TS with master...
I20260928 14:43:03.929649 13251 heartbeater.cc:511] Master 127.9.33.126:43245 requested a full tablet report, sending...
I20260928 14:43:03.930065 12925 ts_manager.cc:194] Registered new tserver with Master: 50956d05116c423cbe5439996042698d (127.9.33.66:40287)
I20260928 14:43:03.930405 12925 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.66@KRBTEST.COM'} at 127.9.33.66:60967
WARNING: no policy specified for kudu/127.9.33.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab.
Entry for principal kudu/127.9.33.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab.
I20260928 14:43:03.951663 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.67:0
--local_ip_for_outbound_sockets=127.9.33.67
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:43245
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:45095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:43:04.038679 13256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:04.038893 13256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:04.038918 13256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:04.040164 13256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:04.040208 13256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:04.040239 13256 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.67
I20260928 14:43:04.041811 13256 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:45095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:43245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13256
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.67
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:04.042078 13256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:04.042330 13256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:04.047823 13262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:04.047927 13261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:04.047823 13264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:04.048053 13256 server_base.cc:1034] running on GCE node
I20260928 14:43:04.048252 13256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:04.048478 13256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:04.049620 13256 hybrid_clock.cc:648] HybridClock initialized: now 1790606584049608 us; error 33 us; skew 500 ppm
Sep 28 14:43:04 dist-test-slave-pv0w krb5kdc[12880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606584, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:04.052322 13256 init.cc:377] Logged in from keytab as kudu/127.9.33.67@KRBTEST.COM (short username kudu)
I20260928 14:43:04.052816 13256 webserver.cc:460] Webserver started at http://127.9.33.67:41511/ using document root <none> and password file <none>
I20260928 14:43:04.053031 13256 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:04.053074 13256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:04.053167 13256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:04.054208 13256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-2/data/instance:
uuid: "775501a04baf4387871f1ab2ed660ea8"
format_stamp: "Formatted at 2026-09-28 14:43:04 on dist-test-slave-pv0w"
server_key: "9833858c84b40ccd14ecad5868a62981"
server_key_iv: "44a8120e3fe351877268eb10d6dfbcc1"
server_key_version: "encryptionkey@0"
I20260928 14:43:04.054517 13256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance:
uuid: "775501a04baf4387871f1ab2ed660ea8"
format_stamp: "Formatted at 2026-09-28 14:43:04 on dist-test-slave-pv0w"
server_key: "9833858c84b40ccd14ecad5868a62981"
server_key_iv: "44a8120e3fe351877268eb10d6dfbcc1"
server_key_version: "encryptionkey@0"
I20260928 14:43:04.056048 13256 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260928 14:43:04.056772 13271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:04.056980 13256 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:04.057077 13256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-2/wal
uuid: "775501a04baf4387871f1ab2ed660ea8"
format_stamp: "Formatted at 2026-09-28 14:43:04 on dist-test-slave-pv0w"
server_key: "9833858c84b40ccd14ecad5868a62981"
server_key_iv: "44a8120e3fe351877268eb10d6dfbcc1"
server_key_version: "encryptionkey@0"
I20260928 14:43:04.057156 13256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:04.072630 13256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:04.075163 13256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:04.075289 13256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:04.075600 13256 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:43:04.075995 13256 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:43:04.076035 13256 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:04.076058 13256 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:43:04.076071 13256 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:04.082233 13256 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.67:41269
I20260928 14:43:04.082278 13384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.67:41269 every 8 connection(s)
I20260928 14:43:04.082616 13256 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
I20260928 14:43:04.086891 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 13256
I20260928 14:43:04.087009 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance
I20260928 14:43:04.087229 9349 external_mini_cluster.cc:1428] Setting key b219afa6ae9e26e73ec68772428c03ab
Sep 28 14:43:04 dist-test-slave-pv0w krb5kdc[12880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606584, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:43:04.092624 13118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:04.083370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:35247 (local address 127.9.33.126:43245)
0928 14:43:04.083418 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:04.083420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:04.083605 (+ 185us) server_negotiation.cc:407] Connection header received
0928 14:43:04.084661 (+ 1056us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:04.084663 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:04.084675 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:04.084733 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:04.085443 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:04.085948 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:04.087192 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:04.087522 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:04.087613 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:04.089900 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:04.089901 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:04.089901 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:04.089908 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:04.091229 (+ 1321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:04.091658 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:04.091658 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:04.091659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:04.091713 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:04.092004 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:04.092004 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:04.092005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:04.092116 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:04.092170 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:04.092482 (+ 312us) server_negotiation.cc:299] Negotiation successful
0928 14:43:04.092554 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 14:43:04.092795 13387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:04.083326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:43245 (local address 127.9.33.67:35247)
0928 14:43:04.083469 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:04.083519 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:04.084542 (+ 1023us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:04.084812 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:04.084817 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:04.085127 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:04.085332 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:04.085336 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:04.086042 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:04.086043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:04.087082 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:04.087086 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:04.087612 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:04.087613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:04.087819 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:04.088431 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:04.088439 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:04.089812 (+ 1373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:04.091306 (+ 1494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:04.091312 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:04.091314 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:04.091587 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:04.091821 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:04.091823 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:04.091823 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:04.091942 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:04.092198 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:04.092204 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:04.092340 (+ 136us) client_negotiation.cc:769] Sending connection context
0928 14:43:04.092440 (+ 100us) client_negotiation.cc:240] Negotiation successful
0928 14:43:04.092574 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":61,"threads_started":1}
I20260928 14:43:04.093284 13385 heartbeater.cc:347] Connected to a master server at 127.9.33.126:43245
I20260928 14:43:04.093405 13385 heartbeater.cc:464] Registering TS with master...
I20260928 14:43:04.093647 13385 heartbeater.cc:511] Master 127.9.33.126:43245 requested a full tablet report, sending...
I20260928 14:43:04.094069 12925 ts_manager.cc:194] Registered new tserver with Master: 775501a04baf4387871f1ab2ed660ea8 (127.9.33.67:41269)
I20260928 14:43:04.094465 12925 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.67@KRBTEST.COM'} at 127.9.33.67:35247
I20260928 14:43:04.099202 9349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 28 14:43:04 dist-test-slave-pv0w krb5kdc[12880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606584, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 14:43:04 dist-test-slave-pv0w krb5kdc[12880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606584, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.65@KRBTEST.COM
I20260928 14:43:04.122658 13392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:04.114589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42336 (local address 127.9.33.65:43413)
0928 14:43:04.114782 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:04.114787 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:04.114798 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:43:04.114825 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:04.114827 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:04.114841 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:04.114911 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:04.115473 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:04.116300 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:04.117051 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:04.117411 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:04.117472 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:04.119774 (+ 2302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:04.119783 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:04.119783 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:04.119790 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:04.121347 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:04.121720 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:04.121747 (+ 27us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:04.121748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:04.121824 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:04.122053 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:04.122054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:04.122054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:04.122208 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:04.122372 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:04.122431 (+ 59us) server_negotiation.cc:299] Negotiation successful
0928 14:43:04.122524 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":99,"threads_started":1}
Sep 28 14:43:04 dist-test-slave-pv0w krb5kdc[12880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606584, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:43:04.131332 13118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:04.124287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46718 (local address 127.9.33.126:43245)
0928 14:43:04.124356 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:04.124358 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:04.124367 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:43:04.124421 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:04.124422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:04.124432 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:04.124483 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:04.124975 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:04.125538 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:04.126230 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:04.126437 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:04.126497 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:04.128840 (+ 2343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:04.128841 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:04.128842 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:04.128850 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:04.130282 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:04.130600 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:04.130601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:04.130602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:04.130652 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:04.130818 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:04.130819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:04.130819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:04.130980 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:04.131059 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:04.131172 (+ 113us) server_negotiation.cc:299] Negotiation successful
0928 14:43:04.131234 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260928 14:43:04.131698 12925 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46718
I20260928 14:43:04.132196 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 12987
I20260928 14:43:04.138383 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 13122
I20260928 14:43:04.142925 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 13256
I20260928 14:43:04.147289 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 12893
2026-09-28T14:43:04Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (5142 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:43:04 dist-test-slave-pv0w krb5kdc[13397](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:43:04 dist-test-slave-pv0w krb5kdc[13397](info): set up 2 sockets
Sep 28 14:43:04 dist-test-slave-pv0w krb5kdc[13397](info): commencing operation
krb5kdc: starting...
W20260928 14:43:05.196614 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:43:05 dist-test-slave-pv0w krb5kdc[13397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:43:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:43:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.33.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
I20260928 14:43:05.325160 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:40791
--webserver_interface=127.9.33.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:34145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:40791
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:43:05.408386 13410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:05.408615 13410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:05.408640 13410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:05.409874 13410 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:43:05.409933 13410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:05.409950 13410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:05.409963 13410 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:43:05.410012 13410 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:43:05.411597 13410 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:34145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:40791
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:40791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:05.411887 13410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:05.412079 13410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 14:43:05.417522 13410 server_base.cc:1034] running on GCE node
W20260928 14:43:05.417497 13415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:05.417497 13416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:05.417488 13418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:05.418004 13410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:05.418394 13410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:05.419574 13410 hybrid_clock.cc:648] HybridClock initialized: now 1790606585419551 us; error 43 us; skew 500 ppm
Sep 28 14:43:05 dist-test-slave-pv0w krb5kdc[13397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606585, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:05.422353 13410 init.cc:377] Logged in from keytab as kudu/127.9.33.126@KRBTEST.COM (short username kudu)
I20260928 14:43:05.422755 13410 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:05.422809 13410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:05.422922 13410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:05.424182 13410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "27fe2eb113994907a9b8412e69eaace1"
format_stamp: "Formatted at 2026-09-28 14:43:05 on dist-test-slave-pv0w"
server_key: "b0a6780d7b58190838d5a702fb591b41"
server_key_iv: "0e859f584b614ee49fab82d71a97a773"
server_key_version: "encryptionkey@0"
I20260928 14:43:05.424535 13410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "27fe2eb113994907a9b8412e69eaace1"
format_stamp: "Formatted at 2026-09-28 14:43:05 on dist-test-slave-pv0w"
server_key: "b0a6780d7b58190838d5a702fb591b41"
server_key_iv: "0e859f584b614ee49fab82d71a97a773"
server_key_version: "encryptionkey@0"
I20260928 14:43:05.426127 13410 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:43:05.426926 13424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:05.427142 13410 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:05.427268 13410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "27fe2eb113994907a9b8412e69eaace1"
format_stamp: "Formatted at 2026-09-28 14:43:05 on dist-test-slave-pv0w"
server_key: "b0a6780d7b58190838d5a702fb591b41"
server_key_iv: "0e859f584b614ee49fab82d71a97a773"
server_key_version: "encryptionkey@0"
I20260928 14:43:05.427351 13410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:05.442970 13410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:05.445487 13410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:05.445675 13410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:05.450146 13410 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:40791
I20260928 14:43:05.450170 13476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:40791 every 8 connection(s)
I20260928 14:43:05.450500 13410 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:43:05.450822 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 13410
I20260928 14:43:05.450938 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:43:05.451133 9349 external_mini_cluster.cc:1428] Setting key 9a8c52275172332212ff8d28d173316b
I20260928 14:43:05.451877 13477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:05.455365 13477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27fe2eb113994907a9b8412e69eaace1: Bootstrap starting.
I20260928 14:43:05.456166 13477 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27fe2eb113994907a9b8412e69eaace1: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:05.456467 13477 log.cc:824] T 00000000000000000000000000000000 P 27fe2eb113994907a9b8412e69eaace1: Log is configured to *not* fsync() on all Append() calls
I20260928 14:43:05.457438 13477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27fe2eb113994907a9b8412e69eaace1: No bootstrap required, opened a new log
Sep 28 14:43:05 dist-test-slave-pv0w krb5kdc[13397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:43:05.459878 13477 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27fe2eb113994907a9b8412e69eaace1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27fe2eb113994907a9b8412e69eaace1" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40791 } }
I20260928 14:43:05.460078 13477 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27fe2eb113994907a9b8412e69eaace1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:05.460121 13477 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27fe2eb113994907a9b8412e69eaace1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27fe2eb113994907a9b8412e69eaace1, State: Initialized, Role: FOLLOWER
I20260928 14:43:05.460279 13477 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27fe2eb113994907a9b8412e69eaace1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27fe2eb113994907a9b8412e69eaace1" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40791 } }
I20260928 14:43:05.460374 13477 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27fe2eb113994907a9b8412e69eaace1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:43:05.460443 13477 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27fe2eb113994907a9b8412e69eaace1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:43:05.460492 13477 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27fe2eb113994907a9b8412e69eaace1 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:05.461076 13477 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27fe2eb113994907a9b8412e69eaace1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27fe2eb113994907a9b8412e69eaace1" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40791 } }
I20260928 14:43:05.461205 13477 leader_election.cc:302] T 00000000000000000000000000000000 P 27fe2eb113994907a9b8412e69eaace1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27fe2eb113994907a9b8412e69eaace1; no voters:
I20260928 14:43:05.461421 13477 leader_election.cc:288] T 00000000000000000000000000000000 P 27fe2eb113994907a9b8412e69eaace1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:43:05.461453 13482 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27fe2eb113994907a9b8412e69eaace1 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:43:05.461587 13482 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27fe2eb113994907a9b8412e69eaace1 [term 1 LEADER]: Becoming Leader. State: Replica: 27fe2eb113994907a9b8412e69eaace1, State: Running, Role: LEADER
I20260928 14:43:05.461735 13477 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27fe2eb113994907a9b8412e69eaace1 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:43:05.461752 13482 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27fe2eb113994907a9b8412e69eaace1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27fe2eb113994907a9b8412e69eaace1" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40791 } }
I20260928 14:43:05.462234 13483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27fe2eb113994907a9b8412e69eaace1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27fe2eb113994907a9b8412e69eaace1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27fe2eb113994907a9b8412e69eaace1" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40791 } } }
I20260928 14:43:05.462335 13483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27fe2eb113994907a9b8412e69eaace1 [sys.catalog]: This master's current role is: LEADER
I20260928 14:43:05.462553 13484 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27fe2eb113994907a9b8412e69eaace1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27fe2eb113994907a9b8412e69eaace1. Latest consensus state: current_term: 1 leader_uuid: "27fe2eb113994907a9b8412e69eaace1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27fe2eb113994907a9b8412e69eaace1" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40791 } } }
I20260928 14:43:05.462664 13484 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27fe2eb113994907a9b8412e69eaace1 [sys.catalog]: This master's current role is: LEADER
W20260928 14:43:05.463665 13498 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 27fe2eb113994907a9b8412e69eaace1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:43:05.463789 13498 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:43:05.463837 13480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:05.451748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59402 (local address 127.9.33.126:40791)
0928 14:43:05.452048 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:05.452058 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:05.452091 (+ 33us) server_negotiation.cc:407] Connection header received
0928 14:43:05.452678 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:05.452688 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:05.452756 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:05.452931 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:05.453590 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:05.455009 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:05.455958 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:05.456610 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:05.456726 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:05.459523 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:05.459529 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:05.459530 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:05.459546 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:05.461524 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:05.461934 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:05.461937 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:05.461937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:05.462048 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:05.462272 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:05.462273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:05.462273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:05.462644 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:05.462796 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:05.463393 (+ 597us) server_negotiation.cc:299] Negotiation successful
0928 14:43:05.463545 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":177,"threads_started":1}
I20260928 14:43:05.464018 13489 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:43:05.464226 13489 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:43:05.465762 13489 catalog_manager.cc:1347] Generated new cluster ID: 41f78de03961450eab790e3a37c1a20d
I20260928 14:43:05.465816 13489 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:43:05.484318 13489 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:43:05.484917 13489 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:43:05.488636 13489 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27fe2eb113994907a9b8412e69eaace1: Generated new TSK 0
I20260928 14:43:05.488849 13489 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.33.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab.
Entry for principal kudu/127.9.33.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab.
I20260928 14:43:05.523422 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.65:0
--local_ip_for_outbound_sockets=127.9.33.65
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:40791
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:34145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:43:05.609405 13503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:05.609624 13503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:05.609650 13503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:05.610857 13503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:05.610899 13503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:05.610926 13503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.65
I20260928 14:43:05.612646 13503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:34145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:40791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.65
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:05.612917 13503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:05.613170 13503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:05.618505 13508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:05.618571 13509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:05.618553 13511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:05.618588 13503 server_base.cc:1034] running on GCE node
I20260928 14:43:05.619016 13503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:05.619215 13503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:05.620409 13503 hybrid_clock.cc:648] HybridClock initialized: now 1790606585620397 us; error 40 us; skew 500 ppm
Sep 28 14:43:05 dist-test-slave-pv0w krb5kdc[13397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606585, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:05.622918 13503 init.cc:377] Logged in from keytab as kudu/127.9.33.65@KRBTEST.COM (short username kudu)
I20260928 14:43:05.623267 13503 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:05.623358 13503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:05.623509 13503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:05.624688 13503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-0/data/instance:
uuid: "098d85e3d2bd4dc28aa8622f2a3e0839"
format_stamp: "Formatted at 2026-09-28 14:43:05 on dist-test-slave-pv0w"
server_key: "9668f9791b421ea3a16374ab3b51d72a"
server_key_iv: "bda500dae2d85dd13939a4fdb7e3cde2"
server_key_version: "encryptionkey@0"
I20260928 14:43:05.625097 13503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance:
uuid: "098d85e3d2bd4dc28aa8622f2a3e0839"
format_stamp: "Formatted at 2026-09-28 14:43:05 on dist-test-slave-pv0w"
server_key: "9668f9791b421ea3a16374ab3b51d72a"
server_key_iv: "bda500dae2d85dd13939a4fdb7e3cde2"
server_key_version: "encryptionkey@0"
I20260928 14:43:05.626560 13503 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:43:05.627287 13517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:05.627463 13503 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 14:43:05.627568 13503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-0/wal
uuid: "098d85e3d2bd4dc28aa8622f2a3e0839"
format_stamp: "Formatted at 2026-09-28 14:43:05 on dist-test-slave-pv0w"
server_key: "9668f9791b421ea3a16374ab3b51d72a"
server_key_iv: "bda500dae2d85dd13939a4fdb7e3cde2"
server_key_version: "encryptionkey@0"
I20260928 14:43:05.627643 13503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:05.638868 13503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:05.641119 13503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:05.641299 13503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:05.641553 13503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:43:05.641891 13503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:43:05.641949 13503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:05.641999 13503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:43:05.642028 13503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:05.648253 13503 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.65:42887
I20260928 14:43:05.648339 13630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.65:42887 every 8 connection(s)
I20260928 14:43:05.648641 13503 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
I20260928 14:43:05.648873 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 13503
I20260928 14:43:05.648995 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance
I20260928 14:43:05.649214 9349 external_mini_cluster.cc:1428] Setting key bc42d353316834898b495e81117bfd00
Sep 28 14:43:05 dist-test-slave-pv0w krb5kdc[13397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606585, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.33.66@KRBTEST.COM; defaulting to no policy
I20260928 14:43:05.658828 13480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:05.649437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:41603 (local address 127.9.33.126:40791)
0928 14:43:05.649507 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:05.649510 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:05.649740 (+ 230us) server_negotiation.cc:407] Connection header received
0928 14:43:05.650376 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:05.650377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:05.650390 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:05.650448 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:05.651533 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:05.652121 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:05.653184 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:05.653402 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:05.653503 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:05.655843 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:05.655845 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:05.655845 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:05.655852 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:05.657288 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:05.657743 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:05.657744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:05.657745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:05.657811 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:05.658086 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:05.658088 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:05.658088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:05.658248 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:05.658331 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:05.658635 (+ 304us) server_negotiation.cc:299] Negotiation successful
0928 14:43:05.658732 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260928 14:43:05.659070 13633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:05.649446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:40791 (local address 127.9.33.65:41603)
0928 14:43:05.649616 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:05.649668 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:05.650262 (+ 594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:05.650557 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:05.650561 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:05.650895 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:05.651422 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:05.651429 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:05.652223 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:05.652225 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:05.653075 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:05.653077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:05.653526 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:05.653527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:05.653661 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:05.654262 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:05.654271 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:05.655674 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:05.657413 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:05.657420 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:05.657423 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:05.657659 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:05.657893 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:05.657894 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:05.657894 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:05.658017 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:05.658331 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:05.658340 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:05.658501 (+ 161us) client_negotiation.cc:769] Sending connection context
0928 14:43:05.658615 (+ 114us) client_negotiation.cc:240] Negotiation successful
0928 14:43:05.658782 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":88,"threads_started":1}
I20260928 14:43:05.659685 13631 heartbeater.cc:347] Connected to a master server at 127.9.33.126:40791
I20260928 14:43:05.659832 13631 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.66@KRBTEST.COM" created.
I20260928 14:43:05.660006 13631 heartbeater.cc:511] Master 127.9.33.126:40791 requested a full tablet report, sending...
I20260928 14:43:05.660629 13441 ts_manager.cc:194] Registered new tserver with Master: 098d85e3d2bd4dc28aa8622f2a3e0839 (127.9.33.65:42887)
I20260928 14:43:05.661638 13441 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.65@KRBTEST.COM'} at 127.9.33.65:41603
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab.
Entry for principal kudu/127.9.33.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab.
I20260928 14:43:05.675885 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.66:0
--local_ip_for_outbound_sockets=127.9.33.66
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:40791
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:34145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:43:05.758251 13636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:05.758493 13636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:05.758520 13636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:05.759814 13636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:05.759867 13636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:05.759903 13636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.66
I20260928 14:43:05.761528 13636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:34145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:40791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.66
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:05.761816 13636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:05.762073 13636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:05.767516 13642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:05.767522 13641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:05.767592 13644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:05.767932 13636 server_base.cc:1034] running on GCE node
I20260928 14:43:05.768240 13636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:05.768546 13636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:05.769713 13636 hybrid_clock.cc:648] HybridClock initialized: now 1790606585769709 us; error 38 us; skew 500 ppm
Sep 28 14:43:05 dist-test-slave-pv0w krb5kdc[13397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606585, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:05.772357 13636 init.cc:377] Logged in from keytab as kudu/127.9.33.66@KRBTEST.COM (short username kudu)
I20260928 14:43:05.772678 13636 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:05.772724 13636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:05.772817 13636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:05.773861 13636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-1/data/instance:
uuid: "0b764ca346ea4213a05d45746cd65b5a"
format_stamp: "Formatted at 2026-09-28 14:43:05 on dist-test-slave-pv0w"
server_key: "5960a0b49400fef2127b43d29a54cc45"
server_key_iv: "29e69e64f9cea0b882262feb0d10e1e8"
server_key_version: "encryptionkey@0"
I20260928 14:43:05.774246 13636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance:
uuid: "0b764ca346ea4213a05d45746cd65b5a"
format_stamp: "Formatted at 2026-09-28 14:43:05 on dist-test-slave-pv0w"
server_key: "5960a0b49400fef2127b43d29a54cc45"
server_key_iv: "29e69e64f9cea0b882262feb0d10e1e8"
server_key_version: "encryptionkey@0"
I20260928 14:43:05.775640 13636 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260928 14:43:05.776595 13650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:05.776782 13636 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:05.776877 13636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-1/wal
uuid: "0b764ca346ea4213a05d45746cd65b5a"
format_stamp: "Formatted at 2026-09-28 14:43:05 on dist-test-slave-pv0w"
server_key: "5960a0b49400fef2127b43d29a54cc45"
server_key_iv: "29e69e64f9cea0b882262feb0d10e1e8"
server_key_version: "encryptionkey@0"
I20260928 14:43:05.776940 13636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:05.793920 13636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:05.796311 13636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:05.796427 13636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:05.796625 13636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:43:05.796882 13636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:43:05.796914 13636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:05.796938 13636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:43:05.796952 13636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:05.802870 13636 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.66:34593
I20260928 14:43:05.802942 13763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.66:34593 every 8 connection(s)
I20260928 14:43:05.803224 13636 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
Sep 28 14:43:05 dist-test-slave-pv0w krb5kdc[13397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606585, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:43:05.810986 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 13636
I20260928 14:43:05.811117 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance
I20260928 14:43:05.811331 9349 external_mini_cluster.cc:1428] Setting key 734a8a9ebe2ad4d8385169f8b07ee66f
I20260928 14:43:05.813552 13480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:05.804051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:33565 (local address 127.9.33.126:40791)
0928 14:43:05.804121 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:05.804123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:05.804331 (+ 208us) server_negotiation.cc:407] Connection header received
0928 14:43:05.804924 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:05.804925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:05.804939 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:05.804997 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:05.806051 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:05.806568 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:05.807681 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:05.807977 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:05.808070 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:05.810497 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:05.810498 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:05.810498 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:05.810508 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:05.812100 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:05.812524 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:05.812525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:05.812525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:05.812586 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:05.812829 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:05.812830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:05.812831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:05.812992 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:05.813060 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:05.813320 (+ 260us) server_negotiation.cc:299] Negotiation successful
0928 14:43:05.813415 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260928 14:43:05.813655 13766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:05.804046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:40791 (local address 127.9.33.66:33565)
0928 14:43:05.804206 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:05.804244 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:05.804794 (+ 550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:05.805078 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:05.805082 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:05.805380 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:05.805926 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:05.805933 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:05.806680 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:05.806681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:05.807573 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:05.807576 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:05.808089 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:05.808090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:05.808240 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:05.808797 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:05.808806 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:05.810391 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:05.812182 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:05.812191 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:05.812193 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:05.812448 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:05.812657 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:05.812658 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:05.812658 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:05.812764 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:05.813057 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:05.813061 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:05.813188 (+ 127us) client_negotiation.cc:769] Sending connection context
0928 14:43:05.813273 (+ 85us) client_negotiation.cc:240] Negotiation successful
0928 14:43:05.813409 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":61,"threads_started":1}
I20260928 14:43:05.815146 13764 heartbeater.cc:347] Connected to a master server at 127.9.33.126:40791
I20260928 14:43:05.815258 13764 heartbeater.cc:464] Registering TS with master...
I20260928 14:43:05.815478 13764 heartbeater.cc:511] Master 127.9.33.126:40791 requested a full tablet report, sending...
I20260928 14:43:05.815940 13441 ts_manager.cc:194] Registered new tserver with Master: 0b764ca346ea4213a05d45746cd65b5a (127.9.33.66:34593)
I20260928 14:43:05.816519 13441 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.66@KRBTEST.COM'} at 127.9.33.66:33565
WARNING: no policy specified for kudu/127.9.33.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab.
Entry for principal kudu/127.9.33.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab.
I20260928 14:43:05.839095 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.67:0
--local_ip_for_outbound_sockets=127.9.33.67
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:40791
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:34145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:43:05.921428 13769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:05.921634 13769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:05.921657 13769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:05.922875 13769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:05.922921 13769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:05.922948 13769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.67
I20260928 14:43:05.924680 13769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:34145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:40791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.67
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:05.924937 13769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:05.925185 13769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:05.930330 13774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:05.930341 13777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:05.930652 13769 server_base.cc:1034] running on GCE node
W20260928 14:43:05.930338 13775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:05.930994 13769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:05.931224 13769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:05.932412 13769 hybrid_clock.cc:648] HybridClock initialized: now 1790606585932396 us; error 29 us; skew 500 ppm
Sep 28 14:43:05 dist-test-slave-pv0w krb5kdc[13397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606585, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:05.935325 13769 init.cc:377] Logged in from keytab as kudu/127.9.33.67@KRBTEST.COM (short username kudu)
I20260928 14:43:05.935793 13769 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:05.935856 13769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:05.935957 13769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:05.937037 13769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-2/data/instance:
uuid: "27736f134bdd43b58b3346e323811c85"
format_stamp: "Formatted at 2026-09-28 14:43:05 on dist-test-slave-pv0w"
server_key: "bc5b1db37d63c2b8cd453c42010bab71"
server_key_iv: "cf2ed1da1b3150a2c8cc282b433c29a5"
server_key_version: "encryptionkey@0"
I20260928 14:43:05.937346 13769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance:
uuid: "27736f134bdd43b58b3346e323811c85"
format_stamp: "Formatted at 2026-09-28 14:43:05 on dist-test-slave-pv0w"
server_key: "bc5b1db37d63c2b8cd453c42010bab71"
server_key_iv: "cf2ed1da1b3150a2c8cc282b433c29a5"
server_key_version: "encryptionkey@0"
I20260928 14:43:05.938663 13769 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:43:05.939440 13783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:05.939692 13769 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:43:05.939836 13769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-2/wal
uuid: "27736f134bdd43b58b3346e323811c85"
format_stamp: "Formatted at 2026-09-28 14:43:05 on dist-test-slave-pv0w"
server_key: "bc5b1db37d63c2b8cd453c42010bab71"
server_key_iv: "cf2ed1da1b3150a2c8cc282b433c29a5"
server_key_version: "encryptionkey@0"
I20260928 14:43:05.939908 13769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:05.957867 13769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:05.960067 13769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:05.960191 13769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:05.960393 13769 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:43:05.960732 13769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:43:05.960768 13769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:05.960791 13769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:43:05.960841 13769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:05.967106 13769 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.67:34823
I20260928 14:43:05.967171 13896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.67:34823 every 8 connection(s)
I20260928 14:43:05.967420 13769 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
Sep 28 14:43:05 dist-test-slave-pv0w krb5kdc[13397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606585, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:43:05.974339 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 13769
I20260928 14:43:05.974463 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance
I20260928 14:43:05.974663 9349 external_mini_cluster.cc:1428] Setting key 967137995749e892e76f16682b21815b
I20260928 14:43:05.976884 13480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:05.968172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:50271 (local address 127.9.33.126:40791)
0928 14:43:05.968253 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:05.968255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:05.968466 (+ 211us) server_negotiation.cc:407] Connection header received
0928 14:43:05.969016 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:05.969018 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:05.969030 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:05.969088 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:05.970033 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:05.970559 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:05.971607 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:05.971897 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:05.971980 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:05.974190 (+ 2210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:05.974191 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:05.974192 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:05.974198 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:05.975688 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:05.976140 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:05.976141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:05.976142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:05.976196 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:05.976390 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:05.976391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:05.976391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:05.976494 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:05.976543 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:05.976751 (+ 208us) server_negotiation.cc:299] Negotiation successful
0928 14:43:05.976809 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260928 14:43:05.977041 13899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:05.968184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:40791 (local address 127.9.33.67:50271)
0928 14:43:05.968348 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:05.968384 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:05.968908 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:05.969165 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:05.969169 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:05.969427 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:05.969926 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:05.969931 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:05.970670 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:05.970671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:05.971501 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:05.971504 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:05.971990 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:05.971991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:05.972114 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:05.972636 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:05.972645 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:05.974097 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:05.975829 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:05.975836 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:05.975837 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:05.976069 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:05.976239 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:05.976240 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:05.976240 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:05.976325 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:05.976533 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:05.976536 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:05.976647 (+ 111us) client_negotiation.cc:769] Sending connection context
0928 14:43:05.976723 (+ 76us) client_negotiation.cc:240] Negotiation successful
0928 14:43:05.976838 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":77,"threads_started":1}
I20260928 14:43:05.977491 13897 heartbeater.cc:347] Connected to a master server at 127.9.33.126:40791
I20260928 14:43:05.977572 13897 heartbeater.cc:464] Registering TS with master...
I20260928 14:43:05.977746 13897 heartbeater.cc:511] Master 127.9.33.126:40791 requested a full tablet report, sending...
I20260928 14:43:05.978173 13441 ts_manager.cc:194] Registered new tserver with Master: 27736f134bdd43b58b3346e323811c85 (127.9.33.67:34823)
I20260928 14:43:05.978502 13441 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.67@KRBTEST.COM'} at 127.9.33.67:50271
I20260928 14:43:05.986548 9349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:43:05.994130 13480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:05.987944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59408 (local address 127.9.33.126:40791)
0928 14:43:05.988016 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:05.988018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:05.988119 (+ 101us) server_negotiation.cc:407] Connection header received
0928 14:43:05.988157 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:05.988158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:05.988166 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:05.988215 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:05.989008 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:05.989493 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:05.990232 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:05.990415 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:05.990472 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:05.991862 (+ 1390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:05.991864 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:05.991864 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:05.991871 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:05.993123 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:05.993428 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:05.993429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:05.993429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:05.993482 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:05.993649 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:05.993650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:05.993650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:05.993782 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:05.993857 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:05.994008 (+ 151us) server_negotiation.cc:299] Negotiation successful
0928 14:43:05.994066 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260928 14:43:05.995573 13441 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59408:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:43:05.997251 13441 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:43:06.004112 13910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:05.999771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44490 (local address 127.9.33.67:34823)
0928 14:43:06.000351 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:06.000356 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:06.000366 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:43:06.000407 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:06.000409 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:06.000425 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:06.000596 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:43:06.000830 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:06.001538 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:06.003046 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:06.003643 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:06.003756 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:06.003821 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:06.003978 (+ 157us) server_negotiation.cc:299] Negotiation successful
0928 14:43:06.004020 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":85,"threads_started":1}
I20260928 14:43:06.004101 13908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:05.999660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:34823 (local address 127.0.0.1:44490)
0928 14:43:05.999825 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:05.999880 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:05.999953 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:06.000691 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:06.000692 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:06.000697 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:43:06.000746 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:06.000748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:06.001643 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:06.001644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:06.002952 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:06.002955 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:06.003759 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:06.003760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:06.003841 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:06.003854 (+ 13us) client_negotiation.cc:769] Sending connection context
0928 14:43:06.003959 (+ 105us) client_negotiation.cc:240] Negotiation successful
0928 14:43:06.004025 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":6,"thread_start_us":44,"threads_started":1}
I20260928 14:43:06.004274 13911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:05.999880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59300 (local address 127.9.33.65:42887)
0928 14:43:06.000067 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:06.000072 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:06.000381 (+ 309us) server_negotiation.cc:407] Connection header received
0928 14:43:06.000517 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:06.000520 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:06.000536 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:06.000614 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:43:06.000837 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:06.001928 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:06.003033 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:06.003742 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:06.003813 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:06.003872 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:06.004140 (+ 268us) server_negotiation.cc:299] Negotiation successful
0928 14:43:06.004184 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":88,"threads_started":1}
I20260928 14:43:06.004347 13909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:05.999661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:34593 (local address 127.0.0.1:56728)
0928 14:43:05.999917 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:05.999929 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:06.000008 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:06.000452 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:06.000455 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:06.000458 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:43:06.000535 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:06.000537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:06.001949 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:06.001950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:06.003194 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:06.003196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:06.004131 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:06.004132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:06.004187 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:06.004190 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:43:06.004220 (+ 30us) client_negotiation.cc:240] Negotiation successful
0928 14:43:06.004252 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":4,"thread_start_us":116,"threads_started":1}
I20260928 14:43:06.004343 13913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:05.999668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:42887 (local address 127.0.0.1:59300)
0928 14:43:06.000309 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:06.000320 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:06.000409 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:06.000706 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:06.000707 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:06.000709 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:43:06.000757 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:06.000758 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:06.002036 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:06.002036 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:06.002942 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:06.002945 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:06.003908 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:06.003909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:06.003986 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:06.003990 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:43:06.004042 (+ 52us) client_negotiation.cc:240] Negotiation successful
0928 14:43:06.004095 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"mutex_wait_us":24,"thread_start_us":587,"threads_started":1}
I20260928 14:43:06.004621 13912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:06.000084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56728 (local address 127.9.33.66:34593)
0928 14:43:06.000235 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:06.000238 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:06.000246 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:43:06.000300 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:06.000302 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:06.000319 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:06.000616 (+ 297us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:43:06.000698 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:06.001843 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:06.003276 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:06.004044 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:06.004390 (+ 346us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:06.004462 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:06.004491 (+ 29us) server_negotiation.cc:299] Negotiation successful
0928 14:43:06.004529 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":77,"threads_started":1}
I20260928 14:43:06.005157 13565 tablet_service.cc:1467] Processing CreateTablet for tablet 92ce805872fe417b9c4a26d3119ed40e (DEFAULT_TABLE table=test-table [id=fa6539492024451aa52b52dfbd9cac59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:43:06.005280 13698 tablet_service.cc:1467] Processing CreateTablet for tablet 92ce805872fe417b9c4a26d3119ed40e (DEFAULT_TABLE table=test-table [id=fa6539492024451aa52b52dfbd9cac59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:43:06.005451 13565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92ce805872fe417b9c4a26d3119ed40e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:06.005652 13698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92ce805872fe417b9c4a26d3119ed40e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:06.008452 13914 tablet_bootstrap.cc:492] T 92ce805872fe417b9c4a26d3119ed40e P 0b764ca346ea4213a05d45746cd65b5a: Bootstrap starting.
I20260928 14:43:06.009126 13914 tablet_bootstrap.cc:654] T 92ce805872fe417b9c4a26d3119ed40e P 0b764ca346ea4213a05d45746cd65b5a: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:06.009483 13831 tablet_service.cc:1467] Processing CreateTablet for tablet 92ce805872fe417b9c4a26d3119ed40e (DEFAULT_TABLE table=test-table [id=fa6539492024451aa52b52dfbd9cac59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:43:06.009833 13914 log.cc:824] T 92ce805872fe417b9c4a26d3119ed40e P 0b764ca346ea4213a05d45746cd65b5a: Log is configured to *not* fsync() on all Append() calls
I20260928 14:43:06.009853 13831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92ce805872fe417b9c4a26d3119ed40e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:06.010212 13915 tablet_bootstrap.cc:492] T 92ce805872fe417b9c4a26d3119ed40e P 098d85e3d2bd4dc28aa8622f2a3e0839: Bootstrap starting.
I20260928 14:43:06.010859 13914 tablet_bootstrap.cc:492] T 92ce805872fe417b9c4a26d3119ed40e P 0b764ca346ea4213a05d45746cd65b5a: No bootstrap required, opened a new log
I20260928 14:43:06.010923 13914 ts_tablet_manager.cc:1397] T 92ce805872fe417b9c4a26d3119ed40e P 0b764ca346ea4213a05d45746cd65b5a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260928 14:43:06.012880 13914 raft_consensus.cc:348] T 92ce805872fe417b9c4a26d3119ed40e P 0b764ca346ea4213a05d45746cd65b5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b764ca346ea4213a05d45746cd65b5a" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 34593 } } peers { permanent_uuid: "27736f134bdd43b58b3346e323811c85" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34823 } } peers { permanent_uuid: "098d85e3d2bd4dc28aa8622f2a3e0839" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 42887 } }
I20260928 14:43:06.013053 13914 raft_consensus.cc:374] T 92ce805872fe417b9c4a26d3119ed40e P 0b764ca346ea4213a05d45746cd65b5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:06.013082 13914 raft_consensus.cc:729] T 92ce805872fe417b9c4a26d3119ed40e P 0b764ca346ea4213a05d45746cd65b5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b764ca346ea4213a05d45746cd65b5a, State: Initialized, Role: FOLLOWER
I20260928 14:43:06.013213 13914 consensus_queue.cc:260] T 92ce805872fe417b9c4a26d3119ed40e P 0b764ca346ea4213a05d45746cd65b5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b764ca346ea4213a05d45746cd65b5a" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 34593 } } peers { permanent_uuid: "27736f134bdd43b58b3346e323811c85" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34823 } } peers { permanent_uuid: "098d85e3d2bd4dc28aa8622f2a3e0839" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 42887 } }
I20260928 14:43:06.013474 13914 ts_tablet_manager.cc:1428] T 92ce805872fe417b9c4a26d3119ed40e P 0b764ca346ea4213a05d45746cd65b5a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:43:06.013582 13764 heartbeater.cc:503] Master 127.9.33.126:40791 was elected leader, sending a full tablet report...
I20260928 14:43:06.013613 13917 tablet_bootstrap.cc:492] T 92ce805872fe417b9c4a26d3119ed40e P 27736f134bdd43b58b3346e323811c85: Bootstrap starting.
I20260928 14:43:06.014297 13917 tablet_bootstrap.cc:654] T 92ce805872fe417b9c4a26d3119ed40e P 27736f134bdd43b58b3346e323811c85: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:06.014922 13915 tablet_bootstrap.cc:654] T 92ce805872fe417b9c4a26d3119ed40e P 098d85e3d2bd4dc28aa8622f2a3e0839: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:06.014935 13917 log.cc:824] T 92ce805872fe417b9c4a26d3119ed40e P 27736f134bdd43b58b3346e323811c85: Log is configured to *not* fsync() on all Append() calls
I20260928 14:43:06.015468 13915 log.cc:824] T 92ce805872fe417b9c4a26d3119ed40e P 098d85e3d2bd4dc28aa8622f2a3e0839: Log is configured to *not* fsync() on all Append() calls
I20260928 14:43:06.015683 13917 tablet_bootstrap.cc:492] T 92ce805872fe417b9c4a26d3119ed40e P 27736f134bdd43b58b3346e323811c85: No bootstrap required, opened a new log
I20260928 14:43:06.015770 13917 ts_tablet_manager.cc:1397] T 92ce805872fe417b9c4a26d3119ed40e P 27736f134bdd43b58b3346e323811c85: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:43:06.016986 13915 tablet_bootstrap.cc:492] T 92ce805872fe417b9c4a26d3119ed40e P 098d85e3d2bd4dc28aa8622f2a3e0839: No bootstrap required, opened a new log
I20260928 14:43:06.017043 13915 ts_tablet_manager.cc:1397] T 92ce805872fe417b9c4a26d3119ed40e P 098d85e3d2bd4dc28aa8622f2a3e0839: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260928 14:43:06.017688 13917 raft_consensus.cc:348] T 92ce805872fe417b9c4a26d3119ed40e P 27736f134bdd43b58b3346e323811c85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b764ca346ea4213a05d45746cd65b5a" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 34593 } } peers { permanent_uuid: "27736f134bdd43b58b3346e323811c85" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34823 } } peers { permanent_uuid: "098d85e3d2bd4dc28aa8622f2a3e0839" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 42887 } }
I20260928 14:43:06.017841 13917 raft_consensus.cc:374] T 92ce805872fe417b9c4a26d3119ed40e P 27736f134bdd43b58b3346e323811c85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:06.017869 13917 raft_consensus.cc:729] T 92ce805872fe417b9c4a26d3119ed40e P 27736f134bdd43b58b3346e323811c85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27736f134bdd43b58b3346e323811c85, State: Initialized, Role: FOLLOWER
I20260928 14:43:06.017976 13917 consensus_queue.cc:260] T 92ce805872fe417b9c4a26d3119ed40e P 27736f134bdd43b58b3346e323811c85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b764ca346ea4213a05d45746cd65b5a" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 34593 } } peers { permanent_uuid: "27736f134bdd43b58b3346e323811c85" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34823 } } peers { permanent_uuid: "098d85e3d2bd4dc28aa8622f2a3e0839" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 42887 } }
I20260928 14:43:06.018215 13917 ts_tablet_manager.cc:1428] T 92ce805872fe417b9c4a26d3119ed40e P 27736f134bdd43b58b3346e323811c85: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:43:06.018340 13897 heartbeater.cc:503] Master 127.9.33.126:40791 was elected leader, sending a full tablet report...
I20260928 14:43:06.018886 13915 raft_consensus.cc:348] T 92ce805872fe417b9c4a26d3119ed40e P 098d85e3d2bd4dc28aa8622f2a3e0839 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b764ca346ea4213a05d45746cd65b5a" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 34593 } } peers { permanent_uuid: "27736f134bdd43b58b3346e323811c85" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34823 } } peers { permanent_uuid: "098d85e3d2bd4dc28aa8622f2a3e0839" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 42887 } }
I20260928 14:43:06.019032 13915 raft_consensus.cc:374] T 92ce805872fe417b9c4a26d3119ed40e P 098d85e3d2bd4dc28aa8622f2a3e0839 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:06.019059 13915 raft_consensus.cc:729] T 92ce805872fe417b9c4a26d3119ed40e P 098d85e3d2bd4dc28aa8622f2a3e0839 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098d85e3d2bd4dc28aa8622f2a3e0839, State: Initialized, Role: FOLLOWER
I20260928 14:43:06.019177 13915 consensus_queue.cc:260] T 92ce805872fe417b9c4a26d3119ed40e P 098d85e3d2bd4dc28aa8622f2a3e0839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b764ca346ea4213a05d45746cd65b5a" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 34593 } } peers { permanent_uuid: "27736f134bdd43b58b3346e323811c85" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34823 } } peers { permanent_uuid: "098d85e3d2bd4dc28aa8622f2a3e0839" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 42887 } }
I20260928 14:43:06.019414 13915 ts_tablet_manager.cc:1428] T 92ce805872fe417b9c4a26d3119ed40e P 098d85e3d2bd4dc28aa8622f2a3e0839: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260928 14:43:06.019481 13631 heartbeater.cc:503] Master 127.9.33.126:40791 was elected leader, sending a full tablet report...
W20260928 14:43:06.053790 13765 tablet.cc:2369] T 92ce805872fe417b9c4a26d3119ed40e P 0b764ca346ea4213a05d45746cd65b5a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:43:06.149422 13632 tablet.cc:2369] T 92ce805872fe417b9c4a26d3119ed40e P 098d85e3d2bd4dc28aa8622f2a3e0839: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:43:06.213186 13921 raft_consensus.cc:482] T 92ce805872fe417b9c4a26d3119ed40e P 27736f134bdd43b58b3346e323811c85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:43:06.213315 13921 raft_consensus.cc:504] T 92ce805872fe417b9c4a26d3119ed40e P 27736f134bdd43b58b3346e323811c85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b764ca346ea4213a05d45746cd65b5a" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 34593 } } peers { permanent_uuid: "27736f134bdd43b58b3346e323811c85" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34823 } } peers { permanent_uuid: "098d85e3d2bd4dc28aa8622f2a3e0839" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 42887 } }
I20260928 14:43:06.213757 13921 leader_election.cc:288] T 92ce805872fe417b9c4a26d3119ed40e P 27736f134bdd43b58b3346e323811c85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b764ca346ea4213a05d45746cd65b5a (127.9.33.66:34593), 098d85e3d2bd4dc28aa8622f2a3e0839 (127.9.33.65:42887)
I20260928 14:43:06.217868 13912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:06.214042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:60735 (local address 127.9.33.66:34593)
0928 14:43:06.214096 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:06.214098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:06.214207 (+ 109us) server_negotiation.cc:407] Connection header received
0928 14:43:06.214283 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:06.214285 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:06.214298 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:06.214379 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:43:06.214551 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:06.215317 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:06.216950 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:06.217510 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:06.217585 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:06.217596 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:06.217756 (+ 160us) server_negotiation.cc:299] Negotiation successful
0928 14:43:06.217793 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260928 14:43:06.217870 13911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:06.214042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:48609 (local address 127.9.33.65:42887)
0928 14:43:06.214089 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:06.214091 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:06.214237 (+ 146us) server_negotiation.cc:407] Connection header received
0928 14:43:06.214285 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:06.214286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:06.214298 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:06.214358 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:43:06.214552 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:06.215332 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:06.216950 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:06.217506 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:06.217585 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:06.217596 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:06.217756 (+ 160us) server_negotiation.cc:299] Negotiation successful
0928 14:43:06.217796 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 14:43:06.217892 13899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:06.213929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:34593 (local address 127.9.33.67:60735)
0928 14:43:06.214128 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:06.214139 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:06.214208 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:06.214385 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:06.214386 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:06.214388 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:43:06.214479 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:06.214481 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:06.215419 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:06.215420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:06.216855 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:06.216859 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:06.217595 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:06.217596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:06.217673 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:06.217680 (+ 7us) client_negotiation.cc:769] Sending connection context
0928 14:43:06.217738 (+ 58us) client_negotiation.cc:240] Negotiation successful
0928 14:43:06.217796 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":105}
I20260928 14:43:06.217898 13923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:06.213983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:42887 (local address 127.9.33.67:48609)
0928 14:43:06.214164 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:06.214172 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:06.214227 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:06.214375 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:06.214376 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:06.214378 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:43:06.214469 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:06.214471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:06.215419 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:06.215420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:06.216856 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:06.216859 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:06.217595 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:06.217596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:06.217673 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:06.217680 (+ 7us) client_negotiation.cc:769] Sending connection context
0928 14:43:06.217738 (+ 58us) client_negotiation.cc:240] Negotiation successful
0928 14:43:06.217795 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":92,"thread_start_us":96,"threads_started":1}
W20260928 14:43:06.218065 13898 tablet.cc:2369] T 92ce805872fe417b9c4a26d3119ed40e P 27736f134bdd43b58b3346e323811c85: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:43:06.218267 13718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92ce805872fe417b9c4a26d3119ed40e" candidate_uuid: "27736f134bdd43b58b3346e323811c85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b764ca346ea4213a05d45746cd65b5a" is_pre_election: true
I20260928 14:43:06.218467 13718 raft_consensus.cc:2454] T 92ce805872fe417b9c4a26d3119ed40e P 0b764ca346ea4213a05d45746cd65b5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27736f134bdd43b58b3346e323811c85 in term 0.
I20260928 14:43:06.218520 13585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92ce805872fe417b9c4a26d3119ed40e" candidate_uuid: "27736f134bdd43b58b3346e323811c85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098d85e3d2bd4dc28aa8622f2a3e0839" is_pre_election: true
I20260928 14:43:06.218696 13585 raft_consensus.cc:2454] T 92ce805872fe417b9c4a26d3119ed40e P 098d85e3d2bd4dc28aa8622f2a3e0839 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27736f134bdd43b58b3346e323811c85 in term 0.
I20260928 14:43:06.218719 13784 leader_election.cc:302] T 92ce805872fe417b9c4a26d3119ed40e P 27736f134bdd43b58b3346e323811c85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b764ca346ea4213a05d45746cd65b5a, 27736f134bdd43b58b3346e323811c85; no voters:
I20260928 14:43:06.218900 13921 raft_consensus.cc:2781] T 92ce805872fe417b9c4a26d3119ed40e P 27736f134bdd43b58b3346e323811c85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:43:06.218957 13921 raft_consensus.cc:482] T 92ce805872fe417b9c4a26d3119ed40e P 27736f134bdd43b58b3346e323811c85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:43:06.218974 13921 raft_consensus.cc:3037] T 92ce805872fe417b9c4a26d3119ed40e P 27736f134bdd43b58b3346e323811c85 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:06.219776 13921 raft_consensus.cc:504] T 92ce805872fe417b9c4a26d3119ed40e P 27736f134bdd43b58b3346e323811c85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b764ca346ea4213a05d45746cd65b5a" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 34593 } } peers { permanent_uuid: "27736f134bdd43b58b3346e323811c85" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34823 } } peers { permanent_uuid: "098d85e3d2bd4dc28aa8622f2a3e0839" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 42887 } }
I20260928 14:43:06.219937 13921 leader_election.cc:288] T 92ce805872fe417b9c4a26d3119ed40e P 27736f134bdd43b58b3346e323811c85 [CANDIDATE]: Term 1 election: Requested vote from peers 0b764ca346ea4213a05d45746cd65b5a (127.9.33.66:34593), 098d85e3d2bd4dc28aa8622f2a3e0839 (127.9.33.65:42887)
I20260928 14:43:06.220072 13718 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92ce805872fe417b9c4a26d3119ed40e" candidate_uuid: "27736f134bdd43b58b3346e323811c85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b764ca346ea4213a05d45746cd65b5a"
I20260928 14:43:06.220156 13718 raft_consensus.cc:3037] T 92ce805872fe417b9c4a26d3119ed40e P 0b764ca346ea4213a05d45746cd65b5a [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:06.220208 13585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92ce805872fe417b9c4a26d3119ed40e" candidate_uuid: "27736f134bdd43b58b3346e323811c85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "098d85e3d2bd4dc28aa8622f2a3e0839"
I20260928 14:43:06.220307 13585 raft_consensus.cc:3037] T 92ce805872fe417b9c4a26d3119ed40e P 098d85e3d2bd4dc28aa8622f2a3e0839 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:06.220935 13718 raft_consensus.cc:2454] T 92ce805872fe417b9c4a26d3119ed40e P 0b764ca346ea4213a05d45746cd65b5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27736f134bdd43b58b3346e323811c85 in term 1.
I20260928 14:43:06.221132 13784 leader_election.cc:302] T 92ce805872fe417b9c4a26d3119ed40e P 27736f134bdd43b58b3346e323811c85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b764ca346ea4213a05d45746cd65b5a, 27736f134bdd43b58b3346e323811c85; no voters:
I20260928 14:43:06.221189 13585 raft_consensus.cc:2454] T 92ce805872fe417b9c4a26d3119ed40e P 098d85e3d2bd4dc28aa8622f2a3e0839 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27736f134bdd43b58b3346e323811c85 in term 1.
I20260928 14:43:06.221243 13921 raft_consensus.cc:2781] T 92ce805872fe417b9c4a26d3119ed40e P 27736f134bdd43b58b3346e323811c85 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:43:06.221434 13921 raft_consensus.cc:686] T 92ce805872fe417b9c4a26d3119ed40e P 27736f134bdd43b58b3346e323811c85 [term 1 LEADER]: Becoming Leader. State: Replica: 27736f134bdd43b58b3346e323811c85, State: Running, Role: LEADER
I20260928 14:43:06.221563 13921 consensus_queue.cc:237] T 92ce805872fe417b9c4a26d3119ed40e P 27736f134bdd43b58b3346e323811c85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b764ca346ea4213a05d45746cd65b5a" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 34593 } } peers { permanent_uuid: "27736f134bdd43b58b3346e323811c85" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34823 } } peers { permanent_uuid: "098d85e3d2bd4dc28aa8622f2a3e0839" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 42887 } }
I20260928 14:43:06.222613 13441 catalog_manager.cc:5495] T 92ce805872fe417b9c4a26d3119ed40e P 27736f134bdd43b58b3346e323811c85 reported cstate change: term changed from 0 to 1, leader changed from <none> to 27736f134bdd43b58b3346e323811c85 (127.9.33.67). New cstate: current_term: 1 leader_uuid: "27736f134bdd43b58b3346e323811c85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b764ca346ea4213a05d45746cd65b5a" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 34593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27736f134bdd43b58b3346e323811c85" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34823 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "098d85e3d2bd4dc28aa8622f2a3e0839" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 42887 } health_report { overall_health: UNKNOWN } } }
I20260928 14:43:06.243037 13910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:06.240477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44492 (local address 127.9.33.67:34823)
0928 14:43:06.240543 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:06.240545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:06.240552 (+ 7us) server_negotiation.cc:407] Connection header received
0928 14:43:06.240585 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:06.240586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:06.240604 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:06.240665 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:43:06.240795 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:06.241251 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:06.242069 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:06.242288 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:06.242355 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:06.242556 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:43:06.242826 (+ 270us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:43:06.242905 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:06.242940 (+ 35us) server_negotiation.cc:299] Negotiation successful
0928 14:43:06.242963 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260928 14:43:06.244954 13585 raft_consensus.cc:1261] T 92ce805872fe417b9c4a26d3119ed40e P 098d85e3d2bd4dc28aa8622f2a3e0839 [term 1 FOLLOWER]: Refusing update from remote peer 27736f134bdd43b58b3346e323811c85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:43:06.244967 13718 raft_consensus.cc:1261] T 92ce805872fe417b9c4a26d3119ed40e P 0b764ca346ea4213a05d45746cd65b5a [term 1 FOLLOWER]: Refusing update from remote peer 27736f134bdd43b58b3346e323811c85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:43:06.245347 13924 consensus_queue.cc:1035] T 92ce805872fe417b9c4a26d3119ed40e P 27736f134bdd43b58b3346e323811c85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "098d85e3d2bd4dc28aa8622f2a3e0839" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 42887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:43:06.245507 13921 consensus_queue.cc:1035] T 92ce805872fe417b9c4a26d3119ed40e P 27736f134bdd43b58b3346e323811c85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b764ca346ea4213a05d45746cd65b5a" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 34593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:43:06.247771 13927 mvcc.cc:205] Tried to move back new op lower bound from 7334324577256218624 to 7334324577164627968. Current Snapshot: MvccSnapshot[applied={T|T < 7334324577256218624}]
I20260928 14:43:06.248821 13929 mvcc.cc:205] Tried to move back new op lower bound from 7334324577256218624 to 7334324577164627968. Current Snapshot: MvccSnapshot[applied={T|T < 7334324577256218624}]
I20260928 14:43:06.250900 13440 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59408:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:43:06.251014 13440 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59408:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:43:06.251955 13440 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 27fe2eb113994907a9b8412e69eaace1: Sending DeleteTablet for 3 replicas of tablet 92ce805872fe417b9c4a26d3119ed40e
I20260928 14:43:06.252341 13831 tablet_service.cc:1514] Processing DeleteTablet for tablet 92ce805872fe417b9c4a26d3119ed40e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:43:06 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:44490
I20260928 14:43:06.252411 13565 tablet_service.cc:1514] Processing DeleteTablet for tablet 92ce805872fe417b9c4a26d3119ed40e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:43:06 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:59300
I20260928 14:43:06.252601 13937 tablet_replica.cc:330] T 92ce805872fe417b9c4a26d3119ed40e P 27736f134bdd43b58b3346e323811c85: stopping tablet replica
I20260928 14:43:06.252601 13698 tablet_service.cc:1514] Processing DeleteTablet for tablet 92ce805872fe417b9c4a26d3119ed40e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:43:06 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:56728
I20260928 14:43:06.252710 13937 raft_consensus.cc:2229] T 92ce805872fe417b9c4a26d3119ed40e P 27736f134bdd43b58b3346e323811c85 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:43:06.252832 13937 raft_consensus.cc:2258] T 92ce805872fe417b9c4a26d3119ed40e P 27736f134bdd43b58b3346e323811c85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:43:06.252861 13939 tablet_replica.cc:330] T 92ce805872fe417b9c4a26d3119ed40e P 0b764ca346ea4213a05d45746cd65b5a: stopping tablet replica
I20260928 14:43:06.252964 13939 raft_consensus.cc:2229] T 92ce805872fe417b9c4a26d3119ed40e P 0b764ca346ea4213a05d45746cd65b5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:43:06.253070 13939 raft_consensus.cc:2258] T 92ce805872fe417b9c4a26d3119ed40e P 0b764ca346ea4213a05d45746cd65b5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:43:06.253319 13937 ts_tablet_manager.cc:1905] T 92ce805872fe417b9c4a26d3119ed40e P 27736f134bdd43b58b3346e323811c85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:43:06.253533 13939 ts_tablet_manager.cc:1905] T 92ce805872fe417b9c4a26d3119ed40e P 0b764ca346ea4213a05d45746cd65b5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:43:06.253579 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 13503
I20260928 14:43:06.255226 13937 ts_tablet_manager.cc:1918] T 92ce805872fe417b9c4a26d3119ed40e P 27736f134bdd43b58b3346e323811c85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:43:06.255292 13937 log.cc:1196] T 92ce805872fe417b9c4a26d3119ed40e P 27736f134bdd43b58b3346e323811c85: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-2/wal/wals/92ce805872fe417b9c4a26d3119ed40e
I20260928 14:43:06.255602 13937 ts_tablet_manager.cc:1939] T 92ce805872fe417b9c4a26d3119ed40e P 27736f134bdd43b58b3346e323811c85: Deleting consensus metadata
I20260928 14:43:06.255599 13939 ts_tablet_manager.cc:1918] T 92ce805872fe417b9c4a26d3119ed40e P 0b764ca346ea4213a05d45746cd65b5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:43:06.255684 13939 log.cc:1196] T 92ce805872fe417b9c4a26d3119ed40e P 0b764ca346ea4213a05d45746cd65b5a: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790606561545179-9349-0/minicluster-data/ts-1/wal/wals/92ce805872fe417b9c4a26d3119ed40e
I20260928 14:43:06.255995 13939 ts_tablet_manager.cc:1939] T 92ce805872fe417b9c4a26d3119ed40e P 0b764ca346ea4213a05d45746cd65b5a: Deleting consensus metadata
I20260928 14:43:06.256052 13428 catalog_manager.cc:4841] TS 27736f134bdd43b58b3346e323811c85 (127.9.33.67:34823): tablet 92ce805872fe417b9c4a26d3119ed40e (table test-table [id=fa6539492024451aa52b52dfbd9cac59]) successfully deleted
I20260928 14:43:06.256588 13425 catalog_manager.cc:4841] TS 0b764ca346ea4213a05d45746cd65b5a (127.9.33.66:34593): tablet 92ce805872fe417b9c4a26d3119ed40e (table test-table [id=fa6539492024451aa52b52dfbd9cac59]) successfully deleted
W20260928 14:43:06.260733 13427 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.33.65:42887 (error 108)
I20260928 14:43:06.261343 13908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:06.261201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:42887 (local address 127.0.0.1:59310)
0928 14:43:06.261261 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:06.261306 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.65:42887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260928 14:43:06.261408 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 13636
W20260928 14:43:06.263857 13427 catalog_manager.cc:4584] TS 098d85e3d2bd4dc28aa8622f2a3e0839 (127.9.33.65:42887): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 92ce805872fe417b9c4a26d3119ed40e: Network error: Client connection negotiation failed: client connection to 127.9.33.65:42887: connect: Connection refused (error 111)
I20260928 14:43:06.269622 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 13769
I20260928 14:43:06.273782 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 13410
2026-09-28T14:43:06Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2125 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-28T14:43:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:43:06Z Disabled control of system clock
I20260928 14:43:06.301798 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:39787
--webserver_interface=127.9.33.126
--webserver_port=0
--builtin_ntp_servers=127.9.33.84:40659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:39787
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260928 14:43:06.388376 13944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:06.388654 13944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:06.388677 13944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:06.389930 13944 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:43:06.389979 13944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:06.389993 13944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:06.390007 13944 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:43:06.390019 13944 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:43:06.391484 13944 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:40659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:39787
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:39787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:06.391796 13944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:06.392123 13944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:06.397231 13949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:06.397253 13950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:06.397223 13952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:06.397517 13944 server_base.cc:1034] running on GCE node
I20260928 14:43:06.397763 13944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:06.398029 13944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:06.399179 13944 hybrid_clock.cc:648] HybridClock initialized: now 1790606586399163 us; error 29 us; skew 500 ppm
I20260928 14:43:06.400561 13944 webserver.cc:460] Webserver started at http://127.9.33.126:43869/ using document root <none> and password file <none>
I20260928 14:43:06.400825 13944 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:06.400898 13944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:06.401086 13944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:06.402248 13944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "7eac25424d464747b1bb3721ab980b14"
format_stamp: "Formatted at 2026-09-28 14:43:06 on dist-test-slave-pv0w"
server_key: "1bcdbd07d3628dc249d647883ee9e278"
server_key_iv: "4140fbd8bde6a6550d32387565c3c9a7"
server_key_version: "encryptionkey@0"
I20260928 14:43:06.402640 13944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "7eac25424d464747b1bb3721ab980b14"
format_stamp: "Formatted at 2026-09-28 14:43:06 on dist-test-slave-pv0w"
server_key: "1bcdbd07d3628dc249d647883ee9e278"
server_key_iv: "4140fbd8bde6a6550d32387565c3c9a7"
server_key_version: "encryptionkey@0"
I20260928 14:43:06.404152 13944 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:43:06.405009 13958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:06.405200 13944 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:43:06.405310 13944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "7eac25424d464747b1bb3721ab980b14"
format_stamp: "Formatted at 2026-09-28 14:43:06 on dist-test-slave-pv0w"
server_key: "1bcdbd07d3628dc249d647883ee9e278"
server_key_iv: "4140fbd8bde6a6550d32387565c3c9a7"
server_key_version: "encryptionkey@0"
I20260928 14:43:06.405408 13944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:06.409044 13944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:06.409340 13944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:06.409493 13944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:06.413647 13944 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:39787
I20260928 14:43:06.413681 14010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:39787 every 8 connection(s)
I20260928 14:43:06.414065 13944 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:43:06.414794 14011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:06.416479 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 13944
I20260928 14:43:06.416594 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:43:06.416786 9349 external_mini_cluster.cc:1428] Setting key 31e7972df948a7e863fc6da214c3c852
I20260928 14:43:06.418493 14011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7eac25424d464747b1bb3721ab980b14: Bootstrap starting.
I20260928 14:43:06.419275 14011 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7eac25424d464747b1bb3721ab980b14: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:06.419634 14011 log.cc:824] T 00000000000000000000000000000000 P 7eac25424d464747b1bb3721ab980b14: Log is configured to *not* fsync() on all Append() calls
I20260928 14:43:06.419709 14014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:06.417396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51564 (local address 127.9.33.126:39787)
0928 14:43:06.417642 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:06.417653 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:06.417689 (+ 36us) server_negotiation.cc:407] Connection header received
0928 14:43:06.418222 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:06.418232 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:06.418292 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:06.418459 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:06.419213 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:06.419217 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:06.419218 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:43:06.419223 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:06.419275 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:43:06.419289 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:06.419387 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:06.419438 (+ 51us) server_negotiation.cc:299] Negotiation successful
0928 14:43:06.419506 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":101,"threads_started":1}
I20260928 14:43:06.420475 14011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7eac25424d464747b1bb3721ab980b14: No bootstrap required, opened a new log
I20260928 14:43:06.421875 14011 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7eac25424d464747b1bb3721ab980b14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eac25424d464747b1bb3721ab980b14" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 39787 } }
I20260928 14:43:06.422048 14011 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7eac25424d464747b1bb3721ab980b14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:06.422098 14011 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7eac25424d464747b1bb3721ab980b14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eac25424d464747b1bb3721ab980b14, State: Initialized, Role: FOLLOWER
I20260928 14:43:06.422226 14011 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7eac25424d464747b1bb3721ab980b14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eac25424d464747b1bb3721ab980b14" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 39787 } }
I20260928 14:43:06.422320 14011 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7eac25424d464747b1bb3721ab980b14 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:43:06.422364 14011 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7eac25424d464747b1bb3721ab980b14 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:43:06.422416 14011 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7eac25424d464747b1bb3721ab980b14 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:06.423267 14011 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7eac25424d464747b1bb3721ab980b14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eac25424d464747b1bb3721ab980b14" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 39787 } }
I20260928 14:43:06.423398 14011 leader_election.cc:302] T 00000000000000000000000000000000 P 7eac25424d464747b1bb3721ab980b14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7eac25424d464747b1bb3721ab980b14; no voters:
I20260928 14:43:06.423619 14011 leader_election.cc:288] T 00000000000000000000000000000000 P 7eac25424d464747b1bb3721ab980b14 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:43:06.423645 14016 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7eac25424d464747b1bb3721ab980b14 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:43:06.423779 14016 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7eac25424d464747b1bb3721ab980b14 [term 1 LEADER]: Becoming Leader. State: Replica: 7eac25424d464747b1bb3721ab980b14, State: Running, Role: LEADER
I20260928 14:43:06.423882 14016 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7eac25424d464747b1bb3721ab980b14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eac25424d464747b1bb3721ab980b14" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 39787 } }
I20260928 14:43:06.423951 14011 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7eac25424d464747b1bb3721ab980b14 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:43:06.424329 14018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7eac25424d464747b1bb3721ab980b14 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7eac25424d464747b1bb3721ab980b14. Latest consensus state: current_term: 1 leader_uuid: "7eac25424d464747b1bb3721ab980b14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eac25424d464747b1bb3721ab980b14" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 39787 } } }
I20260928 14:43:06.424361 14017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7eac25424d464747b1bb3721ab980b14 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7eac25424d464747b1bb3721ab980b14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7eac25424d464747b1bb3721ab980b14" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 39787 } } }
I20260928 14:43:06.424444 14018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7eac25424d464747b1bb3721ab980b14 [sys.catalog]: This master's current role is: LEADER
I20260928 14:43:06.424449 14017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7eac25424d464747b1bb3721ab980b14 [sys.catalog]: This master's current role is: LEADER
I20260928 14:43:06.424765 14024 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:43:06.425623 14024 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:43:06.427182 14024 catalog_manager.cc:1347] Generated new cluster ID: 36abd752cf04428a945fcab334804bd9
I20260928 14:43:06.427239 14024 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:43:06.438874 14024 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:43:06.439108 14024 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:43:06.457746 14024 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7eac25424d464747b1bb3721ab980b14: Generated new TSK 0
I20260928 14:43:06.458001 14024 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:43:06.467142 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.65:0
--local_ip_for_outbound_sockets=127.9.33.65
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:39787
--builtin_ntp_servers=127.9.33.84:40659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260928 14:43:06.550949 14035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:06.551173 14035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:06.551198 14035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:06.552541 14035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:06.552587 14035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:06.552614 14035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.65
I20260928 14:43:06.554024 14035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:40659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:39787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.65
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:06.554298 14035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:06.554625 14035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:06.559945 14040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:06.559937 14041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:06.559998 14035 server_base.cc:1034] running on GCE node
W20260928 14:43:06.559973 14043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:06.560379 14035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:06.560623 14035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:06.561791 14035 hybrid_clock.cc:648] HybridClock initialized: now 1790606586561789 us; error 34 us; skew 500 ppm
I20260928 14:43:06.563115 14035 webserver.cc:460] Webserver started at http://127.9.33.65:45683/ using document root <none> and password file <none>
I20260928 14:43:06.563392 14035 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:06.563483 14035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:06.563638 14035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:06.564800 14035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-0/data/instance:
uuid: "d1dac9d9e00b4b769329710eb3a4d068"
format_stamp: "Formatted at 2026-09-28 14:43:06 on dist-test-slave-pv0w"
server_key: "f9f8521c9ea2b1133439a1adf6b76f97"
server_key_iv: "f36d00df5b40f5883a92b752d940fd7f"
server_key_version: "encryptionkey@0"
I20260928 14:43:06.565177 14035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance:
uuid: "d1dac9d9e00b4b769329710eb3a4d068"
format_stamp: "Formatted at 2026-09-28 14:43:06 on dist-test-slave-pv0w"
server_key: "f9f8521c9ea2b1133439a1adf6b76f97"
server_key_iv: "f36d00df5b40f5883a92b752d940fd7f"
server_key_version: "encryptionkey@0"
I20260928 14:43:06.566500 14035 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:43:06.567188 14049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:06.567411 14035 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:43:06.567533 14035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-0/wal
uuid: "d1dac9d9e00b4b769329710eb3a4d068"
format_stamp: "Formatted at 2026-09-28 14:43:06 on dist-test-slave-pv0w"
server_key: "f9f8521c9ea2b1133439a1adf6b76f97"
server_key_iv: "f36d00df5b40f5883a92b752d940fd7f"
server_key_version: "encryptionkey@0"
I20260928 14:43:06.567611 14035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:06.571067 14035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:06.571339 14035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:06.571489 14035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:06.571696 14035 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:43:06.572037 14035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:43:06.572095 14035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:06.572135 14035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:43:06.572163 14035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:06.577970 14035 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.65:37483
I20260928 14:43:06.578022 14162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.65:37483 every 8 connection(s)
I20260928 14:43:06.578330 14035 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
I20260928 14:43:06.581358 14014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:06.579055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:55381 (local address 127.9.33.126:39787)
0928 14:43:06.579123 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:06.579126 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:06.579342 (+ 216us) server_negotiation.cc:407] Connection header received
0928 14:43:06.579882 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:06.579883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:06.579887 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:06.579955 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:06.581156 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:06.581158 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:06.581158 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:43:06.581162 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:06.581177 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:43:06.581181 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:06.581238 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:06.581293 (+ 55us) server_negotiation.cc:299] Negotiation successful
0928 14:43:06.581305 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260928 14:43:06.581568 14165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:06.579044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:39787 (local address 127.9.33.65:55381)
0928 14:43:06.579213 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:06.579255 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:06.579758 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:06.580012 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:06.580016 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:06.580025 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:06.580973 (+ 948us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:43:06.580974 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:06.581003 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:43:06.581044 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:06.581241 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:06.581245 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:06.581247 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 14:43:06.581301 (+ 54us) client_negotiation.cc:240] Negotiation successful
0928 14:43:06.581354 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":77,"threads_started":1}
I20260928 14:43:06.582064 14163 heartbeater.cc:347] Connected to a master server at 127.9.33.126:39787
I20260928 14:43:06.582167 14163 heartbeater.cc:464] Registering TS with master...
I20260928 14:43:06.582324 14163 heartbeater.cc:511] Master 127.9.33.126:39787 requested a full tablet report, sending...
I20260928 14:43:06.582345 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 14035
I20260928 14:43:06.582436 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance
I20260928 14:43:06.582599 9349 external_mini_cluster.cc:1428] Setting key d3d27836b4889b391e138b87dc9d45bd
I20260928 14:43:06.582931 13975 ts_manager.cc:194] Registered new tserver with Master: d1dac9d9e00b4b769329710eb3a4d068 (127.9.33.65:37483)
I20260928 14:43:06.583853 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.66:0
--local_ip_for_outbound_sockets=127.9.33.66
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:39787
--builtin_ntp_servers=127.9.33.84:40659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260928 14:43:06.666522 14166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:06.666805 14166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:06.666847 14166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:06.668105 14166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:06.668181 14166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:06.668224 14166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.66
I20260928 14:43:06.669657 14166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:40659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:39787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.66
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:06.670006 14166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:06.670360 14166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:06.675505 14174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:06.675513 14172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:06.675474 14171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:06.675675 14166 server_base.cc:1034] running on GCE node
I20260928 14:43:06.675945 14166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:06.676203 14166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:06.677371 14166 hybrid_clock.cc:648] HybridClock initialized: now 1790606586677358 us; error 44 us; skew 500 ppm
I20260928 14:43:06.678663 14166 webserver.cc:460] Webserver started at http://127.9.33.66:43177/ using document root <none> and password file <none>
I20260928 14:43:06.678951 14166 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:06.679030 14166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:06.679189 14166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:06.680406 14166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-1/data/instance:
uuid: "b41275507e3b44f0925cdf786a3eca38"
format_stamp: "Formatted at 2026-09-28 14:43:06 on dist-test-slave-pv0w"
server_key: "973cc6084b16ff4ee7d1c80be15e1e82"
server_key_iv: "376923f95a00425d1d41cf2878c68746"
server_key_version: "encryptionkey@0"
I20260928 14:43:06.680797 14166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance:
uuid: "b41275507e3b44f0925cdf786a3eca38"
format_stamp: "Formatted at 2026-09-28 14:43:06 on dist-test-slave-pv0w"
server_key: "973cc6084b16ff4ee7d1c80be15e1e82"
server_key_iv: "376923f95a00425d1d41cf2878c68746"
server_key_version: "encryptionkey@0"
I20260928 14:43:06.682276 14166 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260928 14:43:06.683053 14180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:06.683243 14166 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:43:06.683352 14166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-1/wal
uuid: "b41275507e3b44f0925cdf786a3eca38"
format_stamp: "Formatted at 2026-09-28 14:43:06 on dist-test-slave-pv0w"
server_key: "973cc6084b16ff4ee7d1c80be15e1e82"
server_key_iv: "376923f95a00425d1d41cf2878c68746"
server_key_version: "encryptionkey@0"
I20260928 14:43:06.683446 14166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:06.686926 14166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:06.687206 14166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:06.687356 14166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:06.687564 14166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:43:06.687965 14166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:43:06.688028 14166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:06.688068 14166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:43:06.688095 14166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:06.694213 14166 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.66:39093
I20260928 14:43:06.694247 14293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.66:39093 every 8 connection(s)
I20260928 14:43:06.694621 14166 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
I20260928 14:43:06.697822 14014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:06.695464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:39757 (local address 127.9.33.126:39787)
0928 14:43:06.695561 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:06.695564 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:06.695787 (+ 223us) server_negotiation.cc:407] Connection header received
0928 14:43:06.696308 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:06.696309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:06.696315 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:06.696382 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:06.697614 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:06.697616 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:06.697616 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:43:06.697625 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:06.697639 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:43:06.697643 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:06.697689 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:06.697755 (+ 66us) server_negotiation.cc:299] Negotiation successful
0928 14:43:06.697767 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260928 14:43:06.698015 14296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:06.695438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:39787 (local address 127.9.33.66:39757)
0928 14:43:06.695644 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:06.695679 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:06.696189 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:06.696445 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:06.696448 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:06.696456 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:06.697458 (+ 1002us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:43:06.697459 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:06.697490 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:43:06.697520 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:06.697695 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:06.697698 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:06.697700 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 14:43:06.697751 (+ 51us) client_negotiation.cc:240] Negotiation successful
0928 14:43:06.697800 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":107,"threads_started":1}
I20260928 14:43:06.698319 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 14166
I20260928 14:43:06.698451 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance
I20260928 14:43:06.698625 14294 heartbeater.cc:347] Connected to a master server at 127.9.33.126:39787
I20260928 14:43:06.698681 9349 external_mini_cluster.cc:1428] Setting key bd16ec22613cd564cdfbe221cb7434a8
I20260928 14:43:06.698727 14294 heartbeater.cc:464] Registering TS with master...
I20260928 14:43:06.698886 14294 heartbeater.cc:511] Master 127.9.33.126:39787 requested a full tablet report, sending...
I20260928 14:43:06.699501 13974 ts_manager.cc:194] Registered new tserver with Master: b41275507e3b44f0925cdf786a3eca38 (127.9.33.66:39093)
I20260928 14:43:06.699761 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.67:0
--local_ip_for_outbound_sockets=127.9.33.67
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:39787
--builtin_ntp_servers=127.9.33.84:40659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260928 14:43:06.785101 14297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:06.785321 14297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:06.785346 14297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:06.786599 14297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:06.786641 14297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:06.786669 14297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.67
I20260928 14:43:06.788187 14297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:40659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:39787
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.67
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:06.788394 14297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:06.788663 14297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:06.794358 14302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:06.794358 14305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:06.794577 14297 server_base.cc:1034] running on GCE node
W20260928 14:43:06.794483 14303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:06.794884 14297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:06.795186 14297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:06.796360 14297 hybrid_clock.cc:648] HybridClock initialized: now 1790606586796332 us; error 40 us; skew 500 ppm
I20260928 14:43:06.797932 14297 webserver.cc:460] Webserver started at http://127.9.33.67:38343/ using document root <none> and password file <none>
I20260928 14:43:06.798228 14297 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:06.798307 14297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:06.798466 14297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:06.799680 14297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-2/data/instance:
uuid: "6828e6d8f7964cdea39168d2555883e9"
format_stamp: "Formatted at 2026-09-28 14:43:06 on dist-test-slave-pv0w"
server_key: "afd157232f56da9e3961bb0d4a72ad99"
server_key_iv: "e0b4aadeb7e8bd243449bbc3d79817fd"
server_key_version: "encryptionkey@0"
I20260928 14:43:06.800156 14297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance:
uuid: "6828e6d8f7964cdea39168d2555883e9"
format_stamp: "Formatted at 2026-09-28 14:43:06 on dist-test-slave-pv0w"
server_key: "afd157232f56da9e3961bb0d4a72ad99"
server_key_iv: "e0b4aadeb7e8bd243449bbc3d79817fd"
server_key_version: "encryptionkey@0"
I20260928 14:43:06.801676 14297 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:43:06.802526 14311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:06.802753 14297 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:06.802870 14297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-2/wal
uuid: "6828e6d8f7964cdea39168d2555883e9"
format_stamp: "Formatted at 2026-09-28 14:43:06 on dist-test-slave-pv0w"
server_key: "afd157232f56da9e3961bb0d4a72ad99"
server_key_iv: "e0b4aadeb7e8bd243449bbc3d79817fd"
server_key_version: "encryptionkey@0"
I20260928 14:43:06.802954 14297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:06.806681 14297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:06.806962 14297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:06.807113 14297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:06.807330 14297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:43:06.807649 14297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:43:06.807708 14297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:06.807770 14297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:43:06.807799 14297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:06.814090 14297 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.67:41165
I20260928 14:43:06.814169 14424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.67:41165 every 8 connection(s)
I20260928 14:43:06.814487 14297 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
I20260928 14:43:06.815054 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 14297
I20260928 14:43:06.815163 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance
I20260928 14:43:06.815416 9349 external_mini_cluster.cc:1428] Setting key 85fb7d09057cf0b4134b9127605887b3
I20260928 14:43:06.818544 14014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:06.815620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:41951 (local address 127.9.33.126:39787)
0928 14:43:06.815751 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:06.815754 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:06.815934 (+ 180us) server_negotiation.cc:407] Connection header received
0928 14:43:06.816598 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:06.816599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:06.816605 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:06.816673 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:06.818145 (+ 1472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:06.818147 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:06.818148 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:43:06.818153 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:06.818171 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:43:06.818176 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:06.818261 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:06.818426 (+ 165us) server_negotiation.cc:299] Negotiation successful
0928 14:43:06.818463 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260928 14:43:06.818750 14427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:06.815554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:39787 (local address 127.9.33.67:41951)
0928 14:43:06.815799 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:06.815848 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:06.816463 (+ 615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:06.816735 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:06.816738 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:06.816746 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:06.817927 (+ 1181us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:43:06.817928 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:06.817973 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:43:06.818013 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:06.818292 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:06.818300 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:06.818303 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:43:06.818434 (+ 131us) client_negotiation.cc:240] Negotiation successful
0928 14:43:06.818515 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":94,"threads_started":1}
I20260928 14:43:06.819350 14425 heartbeater.cc:347] Connected to a master server at 127.9.33.126:39787
I20260928 14:43:06.819463 14425 heartbeater.cc:464] Registering TS with master...
I20260928 14:43:06.819655 14425 heartbeater.cc:511] Master 127.9.33.126:39787 requested a full tablet report, sending...
I20260928 14:43:06.820106 13974 ts_manager.cc:194] Registered new tserver with Master: 6828e6d8f7964cdea39168d2555883e9 (127.9.33.67:41165)
I20260928 14:43:06.827800 9349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:43:06.830463 14014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:06.829477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51578 (local address 127.9.33.126:39787)
0928 14:43:06.829542 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:06.829545 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:06.829599 (+ 54us) server_negotiation.cc:407] Connection header received
0928 14:43:06.829662 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:06.829663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:06.829668 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:06.829721 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:06.830269 (+ 548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:06.830270 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:06.830270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:43:06.830275 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:06.830287 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:43:06.830291 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:06.830337 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:06.830384 (+ 47us) server_negotiation.cc:299] Negotiation successful
0928 14:43:06.830395 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 14:43:06.832337 13974 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51578:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:43:06.833956 13974 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:43:06.838454 14434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:06.836610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:41165 (local address 127.0.0.1:49172)
0928 14:43:06.836815 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:06.836860 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:06.836923 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:06.837320 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:06.837321 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:06.837324 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:06.837942 (+ 618us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:43:06.837942 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:06.837955 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:43:06.837976 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:06.838263 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:06.838271 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:06.838274 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:43:06.838347 (+ 73us) client_negotiation.cc:240] Negotiation successful
0928 14:43:06.838362 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":98,"threads_started":1}
I20260928 14:43:06.838603 14435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:06.836705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49172 (local address 127.9.33.67:41165)
0928 14:43:06.837163 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:06.837168 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:06.837179 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:43:06.837203 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:06.837205 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:06.837213 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:06.837564 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:06.838090 (+ 526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:06.838100 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:06.838100 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:43:06.838107 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:06.838147 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:43:06.838177 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:06.838268 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:06.838366 (+ 98us) server_negotiation.cc:299] Negotiation successful
0928 14:43:06.838484 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":73,"threads_started":1}
I20260928 14:43:06.838869 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:06.836832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:39093 (local address 127.0.0.1:51910)
0928 14:43:06.837010 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:06.837018 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:06.837050 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:06.837649 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:06.837650 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:06.837653 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:06.838166 (+ 513us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:43:06.838167 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:06.838173 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:43:06.838185 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:06.838740 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:06.838741 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:06.838742 (+ 1us) client_negotiation.cc:769] Sending connection context
0928 14:43:06.838772 (+ 30us) client_negotiation.cc:240] Negotiation successful
0928 14:43:06.838786 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":76,"threads_started":1}
I20260928 14:43:06.838999 14436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:06.836717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38202 (local address 127.9.33.65:37483)
0928 14:43:06.836928 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:06.836933 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:06.837098 (+ 165us) server_negotiation.cc:407] Connection header received
0928 14:43:06.837135 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:06.837138 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:06.837148 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:06.837230 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:06.838446 (+ 1216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:06.838456 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:06.838457 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:43:06.838461 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:06.838501 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:43:06.838529 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:06.838722 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:06.838906 (+ 184us) server_negotiation.cc:299] Negotiation successful
0928 14:43:06.838923 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":88,"threads_started":1}
I20260928 14:43:06.839128 14438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:06.837045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51910 (local address 127.9.33.66:39093)
0928 14:43:06.837204 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:06.837209 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:06.837220 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:43:06.837248 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:06.837250 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:06.837259 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:06.837651 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:06.838594 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:06.838603 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:06.838604 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:43:06.838608 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:06.838643 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:43:06.838667 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:06.839000 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:06.839029 (+ 29us) server_negotiation.cc:299] Negotiation successful
0928 14:43:06.839046 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":96,"threads_started":1}
I20260928 14:43:06.839430 14437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:06.836613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:37483 (local address 127.0.0.1:38202)
0928 14:43:06.836971 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:06.836981 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:06.837032 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:06.837303 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:06.837309 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:06.837317 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:06.838345 (+ 1028us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:43:06.838346 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:06.838354 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:43:06.838365 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:06.838744 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:06.838745 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:06.838746 (+ 1us) client_negotiation.cc:769] Sending connection context
0928 14:43:06.838790 (+ 44us) client_negotiation.cc:240] Negotiation successful
0928 14:43:06.838804 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":72,"threads_started":1}
I20260928 14:43:06.839375 14359 tablet_service.cc:1467] Processing CreateTablet for tablet 6de2e705f3984288ae7cc0a8dc2662b5 (DEFAULT_TABLE table=test-table [id=726a03996f034690b586f5b05470cf9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:43:06.839784 14359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6de2e705f3984288ae7cc0a8dc2662b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:06.839805 14228 tablet_service.cc:1467] Processing CreateTablet for tablet 6de2e705f3984288ae7cc0a8dc2662b5 (DEFAULT_TABLE table=test-table [id=726a03996f034690b586f5b05470cf9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:43:06.840122 14228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6de2e705f3984288ae7cc0a8dc2662b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:06.840269 14097 tablet_service.cc:1467] Processing CreateTablet for tablet 6de2e705f3984288ae7cc0a8dc2662b5 (DEFAULT_TABLE table=test-table [id=726a03996f034690b586f5b05470cf9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:43:06.840515 14097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6de2e705f3984288ae7cc0a8dc2662b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:06.843317 14439 tablet_bootstrap.cc:492] T 6de2e705f3984288ae7cc0a8dc2662b5 P d1dac9d9e00b4b769329710eb3a4d068: Bootstrap starting.
I20260928 14:43:06.843387 14440 tablet_bootstrap.cc:492] T 6de2e705f3984288ae7cc0a8dc2662b5 P b41275507e3b44f0925cdf786a3eca38: Bootstrap starting.
I20260928 14:43:06.843387 14441 tablet_bootstrap.cc:492] T 6de2e705f3984288ae7cc0a8dc2662b5 P 6828e6d8f7964cdea39168d2555883e9: Bootstrap starting.
I20260928 14:43:06.844051 14439 tablet_bootstrap.cc:654] T 6de2e705f3984288ae7cc0a8dc2662b5 P d1dac9d9e00b4b769329710eb3a4d068: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:06.844077 14440 tablet_bootstrap.cc:654] T 6de2e705f3984288ae7cc0a8dc2662b5 P b41275507e3b44f0925cdf786a3eca38: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:06.844076 14441 tablet_bootstrap.cc:654] T 6de2e705f3984288ae7cc0a8dc2662b5 P 6828e6d8f7964cdea39168d2555883e9: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:06.844415 14441 log.cc:824] T 6de2e705f3984288ae7cc0a8dc2662b5 P 6828e6d8f7964cdea39168d2555883e9: Log is configured to *not* fsync() on all Append() calls
I20260928 14:43:06.844415 14440 log.cc:824] T 6de2e705f3984288ae7cc0a8dc2662b5 P b41275507e3b44f0925cdf786a3eca38: Log is configured to *not* fsync() on all Append() calls
I20260928 14:43:06.844414 14439 log.cc:824] T 6de2e705f3984288ae7cc0a8dc2662b5 P d1dac9d9e00b4b769329710eb3a4d068: Log is configured to *not* fsync() on all Append() calls
I20260928 14:43:06.845252 14440 tablet_bootstrap.cc:492] T 6de2e705f3984288ae7cc0a8dc2662b5 P b41275507e3b44f0925cdf786a3eca38: No bootstrap required, opened a new log
I20260928 14:43:06.845342 14440 ts_tablet_manager.cc:1397] T 6de2e705f3984288ae7cc0a8dc2662b5 P b41275507e3b44f0925cdf786a3eca38: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260928 14:43:06.845428 14439 tablet_bootstrap.cc:492] T 6de2e705f3984288ae7cc0a8dc2662b5 P d1dac9d9e00b4b769329710eb3a4d068: No bootstrap required, opened a new log
I20260928 14:43:06.845490 14439 ts_tablet_manager.cc:1397] T 6de2e705f3984288ae7cc0a8dc2662b5 P d1dac9d9e00b4b769329710eb3a4d068: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:43:06.847226 14440 raft_consensus.cc:348] T 6de2e705f3984288ae7cc0a8dc2662b5 P b41275507e3b44f0925cdf786a3eca38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6828e6d8f7964cdea39168d2555883e9" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 41165 } } peers { permanent_uuid: "d1dac9d9e00b4b769329710eb3a4d068" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37483 } } peers { permanent_uuid: "b41275507e3b44f0925cdf786a3eca38" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 39093 } }
I20260928 14:43:06.847240 14439 raft_consensus.cc:348] T 6de2e705f3984288ae7cc0a8dc2662b5 P d1dac9d9e00b4b769329710eb3a4d068 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6828e6d8f7964cdea39168d2555883e9" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 41165 } } peers { permanent_uuid: "d1dac9d9e00b4b769329710eb3a4d068" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37483 } } peers { permanent_uuid: "b41275507e3b44f0925cdf786a3eca38" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 39093 } }
I20260928 14:43:06.847389 14439 raft_consensus.cc:374] T 6de2e705f3984288ae7cc0a8dc2662b5 P d1dac9d9e00b4b769329710eb3a4d068 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:06.847393 14440 raft_consensus.cc:374] T 6de2e705f3984288ae7cc0a8dc2662b5 P b41275507e3b44f0925cdf786a3eca38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:06.847442 14440 raft_consensus.cc:729] T 6de2e705f3984288ae7cc0a8dc2662b5 P b41275507e3b44f0925cdf786a3eca38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b41275507e3b44f0925cdf786a3eca38, State: Initialized, Role: FOLLOWER
I20260928 14:43:06.847411 14439 raft_consensus.cc:729] T 6de2e705f3984288ae7cc0a8dc2662b5 P d1dac9d9e00b4b769329710eb3a4d068 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1dac9d9e00b4b769329710eb3a4d068, State: Initialized, Role: FOLLOWER
I20260928 14:43:06.847571 14439 consensus_queue.cc:260] T 6de2e705f3984288ae7cc0a8dc2662b5 P d1dac9d9e00b4b769329710eb3a4d068 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6828e6d8f7964cdea39168d2555883e9" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 41165 } } peers { permanent_uuid: "d1dac9d9e00b4b769329710eb3a4d068" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37483 } } peers { permanent_uuid: "b41275507e3b44f0925cdf786a3eca38" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 39093 } }
I20260928 14:43:06.847572 14440 consensus_queue.cc:260] T 6de2e705f3984288ae7cc0a8dc2662b5 P b41275507e3b44f0925cdf786a3eca38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6828e6d8f7964cdea39168d2555883e9" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 41165 } } peers { permanent_uuid: "d1dac9d9e00b4b769329710eb3a4d068" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37483 } } peers { permanent_uuid: "b41275507e3b44f0925cdf786a3eca38" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 39093 } }
I20260928 14:43:06.847865 14439 ts_tablet_manager.cc:1428] T 6de2e705f3984288ae7cc0a8dc2662b5 P d1dac9d9e00b4b769329710eb3a4d068: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260928 14:43:06.847870 14440 ts_tablet_manager.cc:1428] T 6de2e705f3984288ae7cc0a8dc2662b5 P b41275507e3b44f0925cdf786a3eca38: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260928 14:43:06.848009 14163 heartbeater.cc:503] Master 127.9.33.126:39787 was elected leader, sending a full tablet report...
I20260928 14:43:06.848021 14294 heartbeater.cc:503] Master 127.9.33.126:39787 was elected leader, sending a full tablet report...
I20260928 14:43:06.848135 14441 tablet_bootstrap.cc:492] T 6de2e705f3984288ae7cc0a8dc2662b5 P 6828e6d8f7964cdea39168d2555883e9: No bootstrap required, opened a new log
I20260928 14:43:06.848201 14441 ts_tablet_manager.cc:1397] T 6de2e705f3984288ae7cc0a8dc2662b5 P 6828e6d8f7964cdea39168d2555883e9: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260928 14:43:06.850481 14441 raft_consensus.cc:348] T 6de2e705f3984288ae7cc0a8dc2662b5 P 6828e6d8f7964cdea39168d2555883e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6828e6d8f7964cdea39168d2555883e9" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 41165 } } peers { permanent_uuid: "d1dac9d9e00b4b769329710eb3a4d068" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37483 } } peers { permanent_uuid: "b41275507e3b44f0925cdf786a3eca38" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 39093 } }
I20260928 14:43:06.850621 14441 raft_consensus.cc:374] T 6de2e705f3984288ae7cc0a8dc2662b5 P 6828e6d8f7964cdea39168d2555883e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:06.850644 14441 raft_consensus.cc:729] T 6de2e705f3984288ae7cc0a8dc2662b5 P 6828e6d8f7964cdea39168d2555883e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6828e6d8f7964cdea39168d2555883e9, State: Initialized, Role: FOLLOWER
I20260928 14:43:06.850764 14441 consensus_queue.cc:260] T 6de2e705f3984288ae7cc0a8dc2662b5 P 6828e6d8f7964cdea39168d2555883e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6828e6d8f7964cdea39168d2555883e9" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 41165 } } peers { permanent_uuid: "d1dac9d9e00b4b769329710eb3a4d068" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37483 } } peers { permanent_uuid: "b41275507e3b44f0925cdf786a3eca38" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 39093 } }
I20260928 14:43:06.851066 14441 ts_tablet_manager.cc:1428] T 6de2e705f3984288ae7cc0a8dc2662b5 P 6828e6d8f7964cdea39168d2555883e9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:43:06.851087 14425 heartbeater.cc:503] Master 127.9.33.126:39787 was elected leader, sending a full tablet report...
W20260928 14:43:06.945281 14295 tablet.cc:2369] T 6de2e705f3984288ae7cc0a8dc2662b5 P b41275507e3b44f0925cdf786a3eca38: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:43:07.006400 14446 raft_consensus.cc:482] T 6de2e705f3984288ae7cc0a8dc2662b5 P b41275507e3b44f0925cdf786a3eca38 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:43:07.006527 14446 raft_consensus.cc:504] T 6de2e705f3984288ae7cc0a8dc2662b5 P b41275507e3b44f0925cdf786a3eca38 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6828e6d8f7964cdea39168d2555883e9" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 41165 } } peers { permanent_uuid: "d1dac9d9e00b4b769329710eb3a4d068" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37483 } } peers { permanent_uuid: "b41275507e3b44f0925cdf786a3eca38" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 39093 } }
I20260928 14:43:07.007025 14446 leader_election.cc:288] T 6de2e705f3984288ae7cc0a8dc2662b5 P b41275507e3b44f0925cdf786a3eca38 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6828e6d8f7964cdea39168d2555883e9 (127.9.33.67:41165), d1dac9d9e00b4b769329710eb3a4d068 (127.9.33.65:37483)
I20260928 14:43:07.008689 14435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:07.007445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:59049 (local address 127.9.33.67:41165)
0928 14:43:07.007486 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:07.007488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:07.007505 (+ 17us) server_negotiation.cc:407] Connection header received
0928 14:43:07.007551 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:07.007552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:07.007559 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:07.007620 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:07.008396 (+ 776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:07.008397 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:07.008398 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:43:07.008401 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:07.008417 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:43:07.008421 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:07.008502 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:07.008579 (+ 77us) server_negotiation.cc:299] Negotiation successful
0928 14:43:07.008594 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 14:43:07.008687 14296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:07.007263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:41165 (local address 127.9.33.66:59049)
0928 14:43:07.007343 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:07.007357 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:07.007405 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:07.007701 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:07.007702 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:07.007705 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:07.008288 (+ 583us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:43:07.008289 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:07.008297 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:43:07.008311 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:07.008508 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:07.008508 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:07.008509 (+ 1us) client_negotiation.cc:769] Sending connection context
0928 14:43:07.008585 (+ 76us) client_negotiation.cc:240] Negotiation successful
0928 14:43:07.008600 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260928 14:43:07.008790 14436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:07.007366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:54261 (local address 127.9.33.65:37483)
0928 14:43:07.007433 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:07.007435 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:07.007562 (+ 127us) server_negotiation.cc:407] Connection header received
0928 14:43:07.007589 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:07.007590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:07.007596 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:07.007696 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:07.008497 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:07.008499 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:07.008499 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:43:07.008504 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:07.008527 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:43:07.008531 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:07.008591 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:07.008701 (+ 110us) server_negotiation.cc:299] Negotiation successful
0928 14:43:07.008715 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260928 14:43:07.008896 14448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:07.007283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:37483 (local address 127.9.33.66:54261)
0928 14:43:07.007462 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:07.007471 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:07.007523 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:07.007706 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:07.007706 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:07.007708 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:07.008392 (+ 684us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:43:07.008393 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:07.008397 (+ 4us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:43:07.008407 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:07.008635 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:07.008636 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:07.008636 (+ 0us) client_negotiation.cc:769] Sending connection context
0928 14:43:07.008686 (+ 50us) client_negotiation.cc:240] Negotiation successful
0928 14:43:07.008698 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":97,"threads_started":1}
I20260928 14:43:07.009066 14379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6de2e705f3984288ae7cc0a8dc2662b5" candidate_uuid: "b41275507e3b44f0925cdf786a3eca38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6828e6d8f7964cdea39168d2555883e9" is_pre_election: true
I20260928 14:43:07.009212 14379 raft_consensus.cc:2454] T 6de2e705f3984288ae7cc0a8dc2662b5 P 6828e6d8f7964cdea39168d2555883e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b41275507e3b44f0925cdf786a3eca38 in term 0.
I20260928 14:43:07.009207 14117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6de2e705f3984288ae7cc0a8dc2662b5" candidate_uuid: "b41275507e3b44f0925cdf786a3eca38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1dac9d9e00b4b769329710eb3a4d068" is_pre_election: true
I20260928 14:43:07.009406 14117 raft_consensus.cc:2454] T 6de2e705f3984288ae7cc0a8dc2662b5 P d1dac9d9e00b4b769329710eb3a4d068 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b41275507e3b44f0925cdf786a3eca38 in term 0.
I20260928 14:43:07.009443 14183 leader_election.cc:302] T 6de2e705f3984288ae7cc0a8dc2662b5 P b41275507e3b44f0925cdf786a3eca38 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6828e6d8f7964cdea39168d2555883e9, b41275507e3b44f0925cdf786a3eca38; no voters:
I20260928 14:43:07.009569 14446 raft_consensus.cc:2781] T 6de2e705f3984288ae7cc0a8dc2662b5 P b41275507e3b44f0925cdf786a3eca38 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:43:07.009658 14446 raft_consensus.cc:482] T 6de2e705f3984288ae7cc0a8dc2662b5 P b41275507e3b44f0925cdf786a3eca38 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:43:07.009696 14446 raft_consensus.cc:3037] T 6de2e705f3984288ae7cc0a8dc2662b5 P b41275507e3b44f0925cdf786a3eca38 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:07.010428 14446 raft_consensus.cc:504] T 6de2e705f3984288ae7cc0a8dc2662b5 P b41275507e3b44f0925cdf786a3eca38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6828e6d8f7964cdea39168d2555883e9" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 41165 } } peers { permanent_uuid: "d1dac9d9e00b4b769329710eb3a4d068" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37483 } } peers { permanent_uuid: "b41275507e3b44f0925cdf786a3eca38" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 39093 } }
I20260928 14:43:07.010586 14446 leader_election.cc:288] T 6de2e705f3984288ae7cc0a8dc2662b5 P b41275507e3b44f0925cdf786a3eca38 [CANDIDATE]: Term 1 election: Requested vote from peers 6828e6d8f7964cdea39168d2555883e9 (127.9.33.67:41165), d1dac9d9e00b4b769329710eb3a4d068 (127.9.33.65:37483)
I20260928 14:43:07.010727 14379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6de2e705f3984288ae7cc0a8dc2662b5" candidate_uuid: "b41275507e3b44f0925cdf786a3eca38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6828e6d8f7964cdea39168d2555883e9"
I20260928 14:43:07.010756 14117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6de2e705f3984288ae7cc0a8dc2662b5" candidate_uuid: "b41275507e3b44f0925cdf786a3eca38" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1dac9d9e00b4b769329710eb3a4d068"
I20260928 14:43:07.010828 14117 raft_consensus.cc:3037] T 6de2e705f3984288ae7cc0a8dc2662b5 P d1dac9d9e00b4b769329710eb3a4d068 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:07.010815 14379 raft_consensus.cc:3037] T 6de2e705f3984288ae7cc0a8dc2662b5 P 6828e6d8f7964cdea39168d2555883e9 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:07.011708 14117 raft_consensus.cc:2454] T 6de2e705f3984288ae7cc0a8dc2662b5 P d1dac9d9e00b4b769329710eb3a4d068 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b41275507e3b44f0925cdf786a3eca38 in term 1.
I20260928 14:43:07.011708 14379 raft_consensus.cc:2454] T 6de2e705f3984288ae7cc0a8dc2662b5 P 6828e6d8f7964cdea39168d2555883e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b41275507e3b44f0925cdf786a3eca38 in term 1.
I20260928 14:43:07.011929 14183 leader_election.cc:302] T 6de2e705f3984288ae7cc0a8dc2662b5 P b41275507e3b44f0925cdf786a3eca38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6828e6d8f7964cdea39168d2555883e9, b41275507e3b44f0925cdf786a3eca38; no voters:
I20260928 14:43:07.012036 14446 raft_consensus.cc:2781] T 6de2e705f3984288ae7cc0a8dc2662b5 P b41275507e3b44f0925cdf786a3eca38 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:43:07.012179 14446 raft_consensus.cc:686] T 6de2e705f3984288ae7cc0a8dc2662b5 P b41275507e3b44f0925cdf786a3eca38 [term 1 LEADER]: Becoming Leader. State: Replica: b41275507e3b44f0925cdf786a3eca38, State: Running, Role: LEADER
I20260928 14:43:07.012290 14446 consensus_queue.cc:237] T 6de2e705f3984288ae7cc0a8dc2662b5 P b41275507e3b44f0925cdf786a3eca38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6828e6d8f7964cdea39168d2555883e9" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 41165 } } peers { permanent_uuid: "d1dac9d9e00b4b769329710eb3a4d068" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37483 } } peers { permanent_uuid: "b41275507e3b44f0925cdf786a3eca38" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 39093 } }
I20260928 14:43:07.013118 13975 catalog_manager.cc:5495] T 6de2e705f3984288ae7cc0a8dc2662b5 P b41275507e3b44f0925cdf786a3eca38 reported cstate change: term changed from 0 to 1, leader changed from <none> to b41275507e3b44f0925cdf786a3eca38 (127.9.33.66). New cstate: current_term: 1 leader_uuid: "b41275507e3b44f0925cdf786a3eca38" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6828e6d8f7964cdea39168d2555883e9" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 41165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1dac9d9e00b4b769329710eb3a4d068" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b41275507e3b44f0925cdf786a3eca38" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 39093 } health_report { overall_health: HEALTHY } } }
I20260928 14:43:07.021095 14438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:07.019926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51916 (local address 127.9.33.66:39093)
0928 14:43:07.019988 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:07.019990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:07.020053 (+ 63us) server_negotiation.cc:407] Connection header received
0928 14:43:07.020158 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:07.020159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:07.020176 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:07.020247 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:07.020856 (+ 609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:07.020858 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:07.020858 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:43:07.020863 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:07.020881 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:43:07.020886 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:07.020926 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:07.021007 (+ 81us) server_negotiation.cc:299] Negotiation successful
0928 14:43:07.021020 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260928 14:43:07.022787 14379 raft_consensus.cc:1261] T 6de2e705f3984288ae7cc0a8dc2662b5 P 6828e6d8f7964cdea39168d2555883e9 [term 1 FOLLOWER]: Refusing update from remote peer b41275507e3b44f0925cdf786a3eca38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:43:07.022789 14117 raft_consensus.cc:1261] T 6de2e705f3984288ae7cc0a8dc2662b5 P d1dac9d9e00b4b769329710eb3a4d068 [term 1 FOLLOWER]: Refusing update from remote peer b41275507e3b44f0925cdf786a3eca38: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:43:07.023105 14446 consensus_queue.cc:1035] T 6de2e705f3984288ae7cc0a8dc2662b5 P b41275507e3b44f0925cdf786a3eca38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6828e6d8f7964cdea39168d2555883e9" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 41165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:43:07.023236 14449 consensus_queue.cc:1035] T 6de2e705f3984288ae7cc0a8dc2662b5 P b41275507e3b44f0925cdf786a3eca38 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1dac9d9e00b4b769329710eb3a4d068" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:43:07.026365 14454 mvcc.cc:205] Tried to move back new op lower bound from 7334324580442705920 to 7334324580402917376. Current Snapshot: MvccSnapshot[applied={T|T < 7334324580442705920 or (T in {7334324580442705920})}]
I20260928 14:43:07.026400 14452 mvcc.cc:205] Tried to move back new op lower bound from 7334324580442705920 to 7334324580402917376. Current Snapshot: MvccSnapshot[applied={T|T < 7334324580442705920}]
I20260928 14:43:07.026506 14453 mvcc.cc:205] Tried to move back new op lower bound from 7334324580442705920 to 7334324580402917376. Current Snapshot: MvccSnapshot[applied={T|T < 7334324580442705920 or (T in {7334324580442705920})}]
I20260928 14:43:07.028652 13973 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51578:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:43:07.028769 13973 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51578:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:43:07.029587 13973 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7eac25424d464747b1bb3721ab980b14: Sending DeleteTablet for 3 replicas of tablet 6de2e705f3984288ae7cc0a8dc2662b5
I20260928 14:43:07.029913 14228 tablet_service.cc:1514] Processing DeleteTablet for tablet 6de2e705f3984288ae7cc0a8dc2662b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:43:07 UTC) from {username='slave'} at 127.0.0.1:51910
I20260928 14:43:07.029904 14359 tablet_service.cc:1514] Processing DeleteTablet for tablet 6de2e705f3984288ae7cc0a8dc2662b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:43:07 UTC) from {username='slave'} at 127.0.0.1:49172
I20260928 14:43:07.029951 14097 tablet_service.cc:1514] Processing DeleteTablet for tablet 6de2e705f3984288ae7cc0a8dc2662b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:43:07 UTC) from {username='slave'} at 127.0.0.1:38202
I20260928 14:43:07.030172 14463 tablet_replica.cc:330] T 6de2e705f3984288ae7cc0a8dc2662b5 P b41275507e3b44f0925cdf786a3eca38: stopping tablet replica
I20260928 14:43:07.030282 14463 raft_consensus.cc:2229] T 6de2e705f3984288ae7cc0a8dc2662b5 P b41275507e3b44f0925cdf786a3eca38 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:43:07.030426 14463 raft_consensus.cc:2258] T 6de2e705f3984288ae7cc0a8dc2662b5 P b41275507e3b44f0925cdf786a3eca38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:43:07.030510 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 14035
I20260928 14:43:07.032895 14463 ts_tablet_manager.cc:1905] T 6de2e705f3984288ae7cc0a8dc2662b5 P b41275507e3b44f0925cdf786a3eca38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:43:07.034879 14463 ts_tablet_manager.cc:1918] T 6de2e705f3984288ae7cc0a8dc2662b5 P b41275507e3b44f0925cdf786a3eca38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:43:07.034969 14463 log.cc:1196] T 6de2e705f3984288ae7cc0a8dc2662b5 P b41275507e3b44f0925cdf786a3eca38: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-1/wal/wals/6de2e705f3984288ae7cc0a8dc2662b5
I20260928 14:43:07.035243 14463 ts_tablet_manager.cc:1939] T 6de2e705f3984288ae7cc0a8dc2662b5 P b41275507e3b44f0925cdf786a3eca38: Deleting consensus metadata
I20260928 14:43:07.035815 13962 catalog_manager.cc:4841] TS b41275507e3b44f0925cdf786a3eca38 (127.9.33.66:39093): tablet 6de2e705f3984288ae7cc0a8dc2662b5 (table test-table [id=726a03996f034690b586f5b05470cf9e]) successfully deleted
I20260928 14:43:07.036006 14462 tablet_replica.cc:330] T 6de2e705f3984288ae7cc0a8dc2662b5 P 6828e6d8f7964cdea39168d2555883e9: stopping tablet replica
I20260928 14:43:07.036136 14462 raft_consensus.cc:2229] T 6de2e705f3984288ae7cc0a8dc2662b5 P 6828e6d8f7964cdea39168d2555883e9 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260928 14:43:07.036303 13962 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.33.65:37483 (error 108)
I20260928 14:43:07.036551 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 14166
I20260928 14:43:07.036607 14462 raft_consensus.cc:2258] T 6de2e705f3984288ae7cc0a8dc2662b5 P 6828e6d8f7964cdea39168d2555883e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:43:07.037513 14462 ts_tablet_manager.cc:1905] T 6de2e705f3984288ae7cc0a8dc2662b5 P 6828e6d8f7964cdea39168d2555883e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:43:07.038290 14437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:07.038125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:37483 (local address 127.0.0.1:38208)
0928 14:43:07.038193 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:07.038240 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.65:37483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260928 14:43:07.038437 13962 catalog_manager.cc:4584] TS d1dac9d9e00b4b769329710eb3a4d068 (127.9.33.65:37483): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6de2e705f3984288ae7cc0a8dc2662b5: Network error: Client connection negotiation failed: client connection to 127.9.33.65:37483: connect: Connection refused (error 111)
I20260928 14:43:07.038903 14462 ts_tablet_manager.cc:1918] T 6de2e705f3984288ae7cc0a8dc2662b5 P 6828e6d8f7964cdea39168d2555883e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:43:07.038954 14462 log.cc:1196] T 6de2e705f3984288ae7cc0a8dc2662b5 P 6828e6d8f7964cdea39168d2555883e9: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790606561545179-9349-0/minicluster-data/ts-2/wal/wals/6de2e705f3984288ae7cc0a8dc2662b5
I20260928 14:43:07.039131 14462 ts_tablet_manager.cc:1939] T 6de2e705f3984288ae7cc0a8dc2662b5 P 6828e6d8f7964cdea39168d2555883e9: Deleting consensus metadata
I20260928 14:43:07.039619 13961 catalog_manager.cc:4841] TS 6828e6d8f7964cdea39168d2555883e9 (127.9.33.67:41165): tablet 6de2e705f3984288ae7cc0a8dc2662b5 (table test-table [id=726a03996f034690b586f5b05470cf9e]) successfully deleted
I20260928 14:43:07.041765 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 14297
I20260928 14:43:07.046659 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 13944
2026-09-28T14:43:07Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (772 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260928 14:43:07.066080 9349 mini_oidc.cc:152] Starting JWKS server
I20260928 14:43:07.067210 9349 webserver.cc:460] Webserver started at https://127.0.0.1:42015/ using document root <none> and password file <none>
I20260928 14:43:07.067339 9349 mini_oidc.cc:189] Starting OIDC Discovery server
I20260928 14:43:07.067536 9349 webserver.cc:460] Webserver started at http://127.0.0.1:37625/ using document root <none> and password file <none>
2026-09-28T14:43:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:43:07Z Disabled control of system clock
I20260928 14:43:07.075381 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:37521
--webserver_interface=127.9.33.126
--webserver_port=0
--builtin_ntp_servers=127.9.33.84:42127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:37521
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42015/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606561545179-9349-0/testchainca.cert with env {}
W20260928 14:43:07.158411 14471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:07.158680 14471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:07.158722 14471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:07.159963 14471 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:43:07.160038 14471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:07.160058 14471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:07.160079 14471 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:43:07.160111 14471 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260928 14:43:07.160138 14471 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260928 14:43:07.160164 14471 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42015/jwks.json
I20260928 14:43:07.161764 14471 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:42127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:37521
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:37521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42015/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606561545179-9349-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14471
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:07.162058 14471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:07.162281 14471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:07.167287 14476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:07.167284 14479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:07.167362 14471 server_base.cc:1034] running on GCE node
W20260928 14:43:07.167282 14477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:07.167862 14471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:07.168174 14471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:07.169405 14471 hybrid_clock.cc:648] HybridClock initialized: now 1790606587169379 us; error 44 us; skew 500 ppm
I20260928 14:43:07.170648 14471 webserver.cc:460] Webserver started at http://127.9.33.126:37307/ using document root <none> and password file <none>
I20260928 14:43:07.170909 14471 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:07.170955 14471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:07.171111 14471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:07.172269 14471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "8ad2f241eb4144ad8a533a54a2e95863"
format_stamp: "Formatted at 2026-09-28 14:43:07 on dist-test-slave-pv0w"
server_key: "8430dcf32cd50ad117921242f5b2c236"
server_key_iv: "4c03b9bdae714a05cf9f3923f7209ab4"
server_key_version: "encryptionkey@0"
I20260928 14:43:07.172648 14471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "8ad2f241eb4144ad8a533a54a2e95863"
format_stamp: "Formatted at 2026-09-28 14:43:07 on dist-test-slave-pv0w"
server_key: "8430dcf32cd50ad117921242f5b2c236"
server_key_iv: "4c03b9bdae714a05cf9f3923f7209ab4"
server_key_version: "encryptionkey@0"
I20260928 14:43:07.174041 14471 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260928 14:43:07.174865 14485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:07.175164 14471 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:07.175264 14471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "8ad2f241eb4144ad8a533a54a2e95863"
format_stamp: "Formatted at 2026-09-28 14:43:07 on dist-test-slave-pv0w"
server_key: "8430dcf32cd50ad117921242f5b2c236"
server_key_iv: "4c03b9bdae714a05cf9f3923f7209ab4"
server_key_version: "encryptionkey@0"
I20260928 14:43:07.175339 14471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:07.189760 14471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:07.190096 14471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:07.190248 14471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:07.194226 14471 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:37521
I20260928 14:43:07.194267 14537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:37521 every 8 connection(s)
I20260928 14:43:07.194648 14471 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:43:07.195678 14538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:07.198097 14538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ad2f241eb4144ad8a533a54a2e95863: Bootstrap starting.
I20260928 14:43:07.198747 14538 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ad2f241eb4144ad8a533a54a2e95863: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:07.199072 14538 log.cc:824] T 00000000000000000000000000000000 P 8ad2f241eb4144ad8a533a54a2e95863: Log is configured to *not* fsync() on all Append() calls
I20260928 14:43:07.199958 14538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ad2f241eb4144ad8a533a54a2e95863: No bootstrap required, opened a new log
I20260928 14:43:07.199959 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 14471
I20260928 14:43:07.200100 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:43:07.200290 9349 external_mini_cluster.cc:1428] Setting key ae1af6d906ff20fb3db83868df98e81c
I20260928 14:43:07.201927 14538 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ad2f241eb4144ad8a533a54a2e95863 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ad2f241eb4144ad8a533a54a2e95863" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37521 } }
I20260928 14:43:07.202068 14538 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ad2f241eb4144ad8a533a54a2e95863 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:07.202090 14538 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ad2f241eb4144ad8a533a54a2e95863 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ad2f241eb4144ad8a533a54a2e95863, State: Initialized, Role: FOLLOWER
I20260928 14:43:07.202240 14538 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ad2f241eb4144ad8a533a54a2e95863 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ad2f241eb4144ad8a533a54a2e95863" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37521 } }
I20260928 14:43:07.202347 14538 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ad2f241eb4144ad8a533a54a2e95863 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:43:07.202400 14538 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ad2f241eb4144ad8a533a54a2e95863 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:43:07.202448 14538 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ad2f241eb4144ad8a533a54a2e95863 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:07.203209 14538 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ad2f241eb4144ad8a533a54a2e95863 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ad2f241eb4144ad8a533a54a2e95863" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37521 } }
I20260928 14:43:07.203332 14538 leader_election.cc:302] T 00000000000000000000000000000000 P 8ad2f241eb4144ad8a533a54a2e95863 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ad2f241eb4144ad8a533a54a2e95863; no voters:
I20260928 14:43:07.203508 14538 leader_election.cc:288] T 00000000000000000000000000000000 P 8ad2f241eb4144ad8a533a54a2e95863 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:43:07.203595 14543 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ad2f241eb4144ad8a533a54a2e95863 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:43:07.203778 14538 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ad2f241eb4144ad8a533a54a2e95863 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:43:07.203809 14543 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ad2f241eb4144ad8a533a54a2e95863 [term 1 LEADER]: Becoming Leader. State: Replica: 8ad2f241eb4144ad8a533a54a2e95863, State: Running, Role: LEADER
I20260928 14:43:07.203919 14543 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ad2f241eb4144ad8a533a54a2e95863 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ad2f241eb4144ad8a533a54a2e95863" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37521 } }
I20260928 14:43:07.204262 14544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ad2f241eb4144ad8a533a54a2e95863 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ad2f241eb4144ad8a533a54a2e95863" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ad2f241eb4144ad8a533a54a2e95863" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37521 } } }
I20260928 14:43:07.204368 14544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ad2f241eb4144ad8a533a54a2e95863 [sys.catalog]: This master's current role is: LEADER
I20260928 14:43:07.204262 14543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ad2f241eb4144ad8a533a54a2e95863 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ad2f241eb4144ad8a533a54a2e95863. Latest consensus state: current_term: 1 leader_uuid: "8ad2f241eb4144ad8a533a54a2e95863" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ad2f241eb4144ad8a533a54a2e95863" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37521 } } }
I20260928 14:43:07.204587 14543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ad2f241eb4144ad8a533a54a2e95863 [sys.catalog]: This master's current role is: LEADER
I20260928 14:43:07.204710 14552 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:43:07.205471 14552 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:43:07.207286 14552 catalog_manager.cc:1347] Generated new cluster ID: f3e8ee45dccc4ea6af386fb7a37d3e04
I20260928 14:43:07.207347 14552 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:43:07.208022 14542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:07.200829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59962 (local address 127.9.33.126:37521)
0928 14:43:07.201035 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:07.201043 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:07.201068 (+ 25us) server_negotiation.cc:407] Connection header received
0928 14:43:07.201943 (+ 875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:07.201953 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:07.202027 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:07.202250 (+ 223us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:07.202511 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:07.204359 (+ 1848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:07.205643 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:07.206321 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:07.206442 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:07.207299 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:07.207302 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:07.207303 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:43:07.207530 (+ 227us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:07.207564 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:43:07.207576 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:07.207651 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:07.207754 (+ 103us) server_negotiation.cc:299] Negotiation successful
0928 14:43:07.207837 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":97,"threads_started":1}
I20260928 14:43:07.212755 9349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:43:07.222661 14552 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:43:07.223259 14552 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:43:07.230804 14552 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ad2f241eb4144ad8a533a54a2e95863: Generated new TSK 0
I20260928 14:43:07.231057 14552 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260928 14:43:07.261878 14568 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260928 14:43:07.262043 14568 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260928 14:43:07.262219 14542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:07.237902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59974 (local address 127.9.33.126:37521)
0928 14:43:07.237957 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:07.237960 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:07.238087 (+ 127us) server_negotiation.cc:407] Connection header received
0928 14:43:07.238234 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:07.238235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:07.238248 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:07.238345 (+ 97us) server_negotiation.cc:226] Negotiated authn=JWT
0928 14:43:07.238549 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:07.239127 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:07.240072 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:07.240308 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:07.240407 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:07.240638 (+ 231us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0928 14:43:07.261903 (+ 21265us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42015/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0928 14:43:07.262134 (+ 231us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59974: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42015/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":29}
W20260928 14:43:07.262297 14542 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59974: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42015/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260928 14:43:07.262717 14567 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:07.237804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37521 (local address 127.0.0.1:59974)
0928 14:43:07.237992 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:07.238012 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:07.238110 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:07.238376 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:07.238378 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:07.238385 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0928 14:43:07.238458 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:07.238460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:07.239230 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:07.239231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:07.239975 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:07.239977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:07.240410 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:07.240411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:07.240487 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:07.240508 (+ 21us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0928 14:43:07.262257 (+ 21749us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42015/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0928 14:43:07.262423 (+ 166us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.33.126:37521: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42015/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.9.33.126:37521: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42015/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260928 14:43:07.263298 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 14471
2026-09-28T14:43:07Z chronyd exiting
I20260928 14:43:07.299978 9349 test_util.cc:175] -----------------------------------------------
I20260928 14:43:07.300066 9349 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790606561545179-9349-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (234 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260928 14:43:07.300812 9349 mini_oidc.cc:152] Starting JWKS server
I20260928 14:43:07.301687 9349 webserver.cc:460] Webserver started at https://127.0.0.1:37849/ using document root <none> and password file <none>
I20260928 14:43:07.301733 9349 mini_oidc.cc:189] Starting OIDC Discovery server
I20260928 14:43:07.301928 9349 webserver.cc:460] Webserver started at http://127.0.0.1:39047/ using document root <none> and password file <none>
2026-09-28T14:43:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:43:07Z Disabled control of system clock
I20260928 14:43:07.309446 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:38239
--webserver_interface=127.9.33.126
--webserver_port=0
--builtin_ntp_servers=127.9.33.84:42169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:38239
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37849/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790606561545179-9349-0/testchainca.cert with env {}
W20260928 14:43:07.389042 14574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:07.389276 14574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:07.389313 14574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:07.390453 14574 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:43:07.390524 14574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:07.390545 14574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:07.390564 14574 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:43:07.390582 14574 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260928 14:43:07.390611 14574 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260928 14:43:07.390635 14574 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37849/jwks.json
I20260928 14:43:07.392246 14574 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:42169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:38239
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:38239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37849/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790606561545179-9349-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:07.392530 14574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:07.392774 14574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:07.397665 14579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:07.397689 14580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:07.397871 14574 server_base.cc:1034] running on GCE node
W20260928 14:43:07.397677 14582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:07.398169 14574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:07.398452 14574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:07.399642 14574 hybrid_clock.cc:648] HybridClock initialized: now 1790606587399617 us; error 35 us; skew 500 ppm
I20260928 14:43:07.401074 14574 webserver.cc:460] Webserver started at http://127.9.33.126:40489/ using document root <none> and password file <none>
I20260928 14:43:07.401360 14574 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:07.401413 14574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:07.401608 14574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:07.402835 14574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "fa3f22eb9a67459a9f7d80ad670d9d76"
format_stamp: "Formatted at 2026-09-28 14:43:07 on dist-test-slave-pv0w"
server_key: "dca18bbe5b594809f5fd68998f1a7286"
server_key_iv: "b5f007351943802571100b8a5c5d9a12"
server_key_version: "encryptionkey@0"
I20260928 14:43:07.403241 14574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "fa3f22eb9a67459a9f7d80ad670d9d76"
format_stamp: "Formatted at 2026-09-28 14:43:07 on dist-test-slave-pv0w"
server_key: "dca18bbe5b594809f5fd68998f1a7286"
server_key_iv: "b5f007351943802571100b8a5c5d9a12"
server_key_version: "encryptionkey@0"
I20260928 14:43:07.404815 14574 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:43:07.405650 14588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:07.405839 14574 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:43:07.405993 14574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "fa3f22eb9a67459a9f7d80ad670d9d76"
format_stamp: "Formatted at 2026-09-28 14:43:07 on dist-test-slave-pv0w"
server_key: "dca18bbe5b594809f5fd68998f1a7286"
server_key_iv: "b5f007351943802571100b8a5c5d9a12"
server_key_version: "encryptionkey@0"
I20260928 14:43:07.406075 14574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:07.423295 14574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:07.423635 14574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:07.423817 14574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:07.427928 14574 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:38239
I20260928 14:43:07.427961 14640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:38239 every 8 connection(s)
I20260928 14:43:07.428319 14574 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:43:07.429298 14641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:07.431615 14641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa3f22eb9a67459a9f7d80ad670d9d76: Bootstrap starting.
I20260928 14:43:07.432225 14641 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa3f22eb9a67459a9f7d80ad670d9d76: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:07.432459 14641 log.cc:824] T 00000000000000000000000000000000 P fa3f22eb9a67459a9f7d80ad670d9d76: Log is configured to *not* fsync() on all Append() calls
I20260928 14:43:07.433281 14641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa3f22eb9a67459a9f7d80ad670d9d76: No bootstrap required, opened a new log
I20260928 14:43:07.433764 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 14574
I20260928 14:43:07.433876 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:43:07.434054 9349 external_mini_cluster.cc:1428] Setting key f68ba19471736223dfd742b3a53058ac
I20260928 14:43:07.435664 14641 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa3f22eb9a67459a9f7d80ad670d9d76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa3f22eb9a67459a9f7d80ad670d9d76" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38239 } }
I20260928 14:43:07.435861 14641 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa3f22eb9a67459a9f7d80ad670d9d76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:07.435890 14641 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa3f22eb9a67459a9f7d80ad670d9d76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa3f22eb9a67459a9f7d80ad670d9d76, State: Initialized, Role: FOLLOWER
I20260928 14:43:07.436028 14641 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa3f22eb9a67459a9f7d80ad670d9d76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa3f22eb9a67459a9f7d80ad670d9d76" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38239 } }
I20260928 14:43:07.436102 14641 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa3f22eb9a67459a9f7d80ad670d9d76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:43:07.436124 14641 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa3f22eb9a67459a9f7d80ad670d9d76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:43:07.436170 14641 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa3f22eb9a67459a9f7d80ad670d9d76 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:07.436738 14641 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa3f22eb9a67459a9f7d80ad670d9d76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa3f22eb9a67459a9f7d80ad670d9d76" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38239 } }
I20260928 14:43:07.436827 14641 leader_election.cc:302] T 00000000000000000000000000000000 P fa3f22eb9a67459a9f7d80ad670d9d76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa3f22eb9a67459a9f7d80ad670d9d76; no voters:
I20260928 14:43:07.437054 14641 leader_election.cc:288] T 00000000000000000000000000000000 P fa3f22eb9a67459a9f7d80ad670d9d76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:43:07.437076 14646 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa3f22eb9a67459a9f7d80ad670d9d76 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:43:07.437194 14646 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa3f22eb9a67459a9f7d80ad670d9d76 [term 1 LEADER]: Becoming Leader. State: Replica: fa3f22eb9a67459a9f7d80ad670d9d76, State: Running, Role: LEADER
I20260928 14:43:07.437337 14641 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa3f22eb9a67459a9f7d80ad670d9d76 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:43:07.437335 14646 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa3f22eb9a67459a9f7d80ad670d9d76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa3f22eb9a67459a9f7d80ad670d9d76" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38239 } }
I20260928 14:43:07.437849 14647 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa3f22eb9a67459a9f7d80ad670d9d76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa3f22eb9a67459a9f7d80ad670d9d76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa3f22eb9a67459a9f7d80ad670d9d76" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38239 } } }
I20260928 14:43:07.438102 14647 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa3f22eb9a67459a9f7d80ad670d9d76 [sys.catalog]: This master's current role is: LEADER
I20260928 14:43:07.438024 14648 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa3f22eb9a67459a9f7d80ad670d9d76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa3f22eb9a67459a9f7d80ad670d9d76. Latest consensus state: current_term: 1 leader_uuid: "fa3f22eb9a67459a9f7d80ad670d9d76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa3f22eb9a67459a9f7d80ad670d9d76" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38239 } } }
I20260928 14:43:07.438544 14648 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa3f22eb9a67459a9f7d80ad670d9d76 [sys.catalog]: This master's current role is: LEADER
I20260928 14:43:07.438763 14655 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:43:07.439421 14655 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:43:07.441205 14655 catalog_manager.cc:1347] Generated new cluster ID: 92756c2c0fac4d5f97e9f0c4c3537bc1
I20260928 14:43:07.441267 14655 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:43:07.441362 14645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:07.434565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54218 (local address 127.9.33.126:38239)
0928 14:43:07.434766 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:07.434773 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:07.434798 (+ 25us) server_negotiation.cc:407] Connection header received
0928 14:43:07.435206 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:07.435213 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:07.435274 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:07.435395 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:07.435676 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:07.437548 (+ 1872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:07.439014 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:07.439596 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:07.439764 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:07.440531 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:07.440535 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:07.440536 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:43:07.440767 (+ 231us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:07.440812 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:43:07.440828 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:07.440903 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:07.440996 (+ 93us) server_negotiation.cc:299] Negotiation successful
0928 14:43:07.441085 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":77,"threads_started":1}
I20260928 14:43:07.442334 9349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:43:07.458793 14655 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:43:07.459352 14655 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:43:07.464376 14655 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa3f22eb9a67459a9f7d80ad670d9d76: Generated new TSK 0
I20260928 14:43:07.464576 14655 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260928 14:43:07.493606 14671 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260928 14:43:07.493702 14671 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260928 14:43:07.494040 14645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:07.479451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54234 (local address 127.9.33.126:38239)
0928 14:43:07.479495 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:07.479497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:07.479605 (+ 108us) server_negotiation.cc:407] Connection header received
0928 14:43:07.479660 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:07.479661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:07.479673 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:07.479799 (+ 126us) server_negotiation.cc:226] Negotiated authn=JWT
0928 14:43:07.479931 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:07.480737 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:07.481500 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:07.481841 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:07.481914 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:07.482088 (+ 174us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0928 14:43:07.493812 (+ 11724us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37849/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0928 14:43:07.493967 (+ 155us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54234: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37849/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":17}
W20260928 14:43:07.494103 14645 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54234: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37849/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260928 14:43:07.494141 14670 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:07.479338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38239 (local address 127.0.0.1:54234)
0928 14:43:07.479527 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:07.479537 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:07.479602 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:07.479801 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:07.479802 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:07.479804 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0928 14:43:07.479865 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:07.479867 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:07.480817 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:07.480818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:07.481418 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:07.481421 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:07.481922 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:07.481923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:07.481982 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:07.481987 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0928 14:43:07.493931 (+ 11944us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37849/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0928 14:43:07.494037 (+ 106us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.33.126:38239: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37849/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":110,"threads_started":1}
I20260928 14:43:07.494526 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 14574
2026-09-28T14:43:07Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (231 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260928 14:43:07.532485 9349 mini_oidc.cc:152] Starting JWKS server
I20260928 14:43:07.533154 9349 webserver.cc:460] Webserver started at https://127.0.0.1:38197/ using document root <none> and password file <none>
I20260928 14:43:07.533192 9349 mini_oidc.cc:189] Starting OIDC Discovery server
I20260928 14:43:07.533335 9349 webserver.cc:460] Webserver started at http://127.0.0.1:44241/ using document root <none> and password file <none>
2026-09-28T14:43:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:43:07Z Disabled control of system clock
I20260928 14:43:07.541152 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:34961
--webserver_interface=127.9.33.126
--webserver_port=0
--builtin_ntp_servers=127.9.33.84:40549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:34961
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38197/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260928 14:43:07.623015 14677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:07.623281 14677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:07.623324 14677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:07.624598 14677 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:43:07.624675 14677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:07.624696 14677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:07.624718 14677 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:43:07.624774 14677 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260928 14:43:07.624805 14677 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260928 14:43:07.624830 14677 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38197/jwks.json
I20260928 14:43:07.626319 14677 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:40549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:34961
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:34961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38197/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14677
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:07.626618 14677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:07.626940 14677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:07.632027 14685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:07.632038 14683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:07.632355 14677 server_base.cc:1034] running on GCE node
W20260928 14:43:07.632054 14682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:07.632656 14677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:07.632964 14677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:07.634131 14677 hybrid_clock.cc:648] HybridClock initialized: now 1790606587634130 us; error 41 us; skew 500 ppm
I20260928 14:43:07.635438 14677 webserver.cc:460] Webserver started at http://127.9.33.126:43259/ using document root <none> and password file <none>
I20260928 14:43:07.635689 14677 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:07.635782 14677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:07.635943 14677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:07.637117 14677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "e6298c81191a4fb1918811b1b1972620"
format_stamp: "Formatted at 2026-09-28 14:43:07 on dist-test-slave-pv0w"
server_key: "99291d68aaad9d599e2b2b3713c2e989"
server_key_iv: "1166bb759cea54ad6fdd75d4276017e9"
server_key_version: "encryptionkey@0"
I20260928 14:43:07.637519 14677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "e6298c81191a4fb1918811b1b1972620"
format_stamp: "Formatted at 2026-09-28 14:43:07 on dist-test-slave-pv0w"
server_key: "99291d68aaad9d599e2b2b3713c2e989"
server_key_iv: "1166bb759cea54ad6fdd75d4276017e9"
server_key_version: "encryptionkey@0"
I20260928 14:43:07.638881 14677 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:43:07.639657 14691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:07.639844 14677 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:43:07.639954 14677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "e6298c81191a4fb1918811b1b1972620"
format_stamp: "Formatted at 2026-09-28 14:43:07 on dist-test-slave-pv0w"
server_key: "99291d68aaad9d599e2b2b3713c2e989"
server_key_iv: "1166bb759cea54ad6fdd75d4276017e9"
server_key_version: "encryptionkey@0"
I20260928 14:43:07.640038 14677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:07.664935 14677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:07.665307 14677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:07.665478 14677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:07.669659 14677 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:34961
I20260928 14:43:07.669696 14743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:34961 every 8 connection(s)
I20260928 14:43:07.670081 14677 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:43:07.670828 14744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:07.673807 14744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6298c81191a4fb1918811b1b1972620: Bootstrap starting.
I20260928 14:43:07.674420 14744 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6298c81191a4fb1918811b1b1972620: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:07.674721 14744 log.cc:824] T 00000000000000000000000000000000 P e6298c81191a4fb1918811b1b1972620: Log is configured to *not* fsync() on all Append() calls
I20260928 14:43:07.675426 14744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6298c81191a4fb1918811b1b1972620: No bootstrap required, opened a new log
I20260928 14:43:07.675832 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 14677
I20260928 14:43:07.675915 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:43:07.676106 9349 external_mini_cluster.cc:1428] Setting key b30337428087b773b401011d39e8c3a3
I20260928 14:43:07.677843 14744 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6298c81191a4fb1918811b1b1972620 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6298c81191a4fb1918811b1b1972620" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 34961 } }
I20260928 14:43:07.678042 14744 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6298c81191a4fb1918811b1b1972620 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:07.678085 14744 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6298c81191a4fb1918811b1b1972620 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6298c81191a4fb1918811b1b1972620, State: Initialized, Role: FOLLOWER
I20260928 14:43:07.678241 14744 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6298c81191a4fb1918811b1b1972620 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6298c81191a4fb1918811b1b1972620" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 34961 } }
I20260928 14:43:07.678321 14744 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6298c81191a4fb1918811b1b1972620 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:43:07.678354 14744 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6298c81191a4fb1918811b1b1972620 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:43:07.678400 14744 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6298c81191a4fb1918811b1b1972620 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:07.679106 14744 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6298c81191a4fb1918811b1b1972620 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6298c81191a4fb1918811b1b1972620" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 34961 } }
I20260928 14:43:07.679214 14744 leader_election.cc:302] T 00000000000000000000000000000000 P e6298c81191a4fb1918811b1b1972620 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6298c81191a4fb1918811b1b1972620; no voters:
I20260928 14:43:07.679374 14744 leader_election.cc:288] T 00000000000000000000000000000000 P e6298c81191a4fb1918811b1b1972620 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:43:07.679528 14750 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6298c81191a4fb1918811b1b1972620 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:43:07.679590 14744 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6298c81191a4fb1918811b1b1972620 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:43:07.679674 14750 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6298c81191a4fb1918811b1b1972620 [term 1 LEADER]: Becoming Leader. State: Replica: e6298c81191a4fb1918811b1b1972620, State: Running, Role: LEADER
I20260928 14:43:07.679793 14750 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6298c81191a4fb1918811b1b1972620 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6298c81191a4fb1918811b1b1972620" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 34961 } }
I20260928 14:43:07.680128 14750 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6298c81191a4fb1918811b1b1972620 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6298c81191a4fb1918811b1b1972620" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6298c81191a4fb1918811b1b1972620" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 34961 } } }
I20260928 14:43:07.680258 14750 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6298c81191a4fb1918811b1b1972620 [sys.catalog]: This master's current role is: LEADER
I20260928 14:43:07.680430 14750 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6298c81191a4fb1918811b1b1972620 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6298c81191a4fb1918811b1b1972620. Latest consensus state: current_term: 1 leader_uuid: "e6298c81191a4fb1918811b1b1972620" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6298c81191a4fb1918811b1b1972620" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 34961 } } }
I20260928 14:43:07.680549 14750 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6298c81191a4fb1918811b1b1972620 [sys.catalog]: This master's current role is: LEADER
I20260928 14:43:07.680868 14758 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:43:07.681483 14758 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:43:07.683022 14748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:07.676664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42868 (local address 127.9.33.126:34961)
0928 14:43:07.676884 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:07.676895 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:07.676929 (+ 34us) server_negotiation.cc:407] Connection header received
0928 14:43:07.677402 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:07.677411 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:07.677494 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:07.677655 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:07.677814 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:07.679586 (+ 1772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:07.680829 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:07.681440 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:07.681582 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:07.682262 (+ 680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:07.682266 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:07.682266 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:43:07.682486 (+ 220us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:07.682528 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:43:07.682545 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:07.682613 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:07.682735 (+ 122us) server_negotiation.cc:299] Negotiation successful
0928 14:43:07.682815 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":81,"threads_started":1}
I20260928 14:43:07.683151 14758 catalog_manager.cc:1347] Generated new cluster ID: 7b127a794f024e8c955bf4d524228062
I20260928 14:43:07.683211 14758 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:43:07.684005 9349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:43:07.692022 14758 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:43:07.692553 14758 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:43:07.696076 14758 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6298c81191a4fb1918811b1b1972620: Generated new TSK 0
I20260928 14:43:07.696249 14758 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:43:07.708983 14748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:07.705521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42880 (local address 127.9.33.126:34961)
0928 14:43:07.705585 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:07.705587 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:07.705706 (+ 119us) server_negotiation.cc:407] Connection header received
0928 14:43:07.705793 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:07.705794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:07.705805 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:07.705896 (+ 91us) server_negotiation.cc:226] Negotiated authn=JWT
0928 14:43:07.706022 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:07.706526 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:07.707328 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:07.707540 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:07.707616 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:07.707839 (+ 223us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0928 14:43:07.708766 (+ 927us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38197/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0928 14:43:07.708916 (+ 150us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42880: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38197/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":39}
W20260928 14:43:07.709108 14773 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:07.705471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:34961 (local address 127.0.0.1:42880)
0928 14:43:07.705635 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:07.705645 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:07.705723 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:07.705899 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:07.705900 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:07.705901 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0928 14:43:07.705954 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:07.705956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:07.706626 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:07.706627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:07.707225 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:07.707227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:07.707623 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:07.707624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:07.707686 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:07.707700 (+ 14us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0928 14:43:07.708894 (+ 1194us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38197/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0928 14:43:07.709007 (+ 113us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.33.126:34961: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38197/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":100,"threads_started":1}
W20260928 14:43:07.709255 14748 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42880: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38197/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260928 14:43:07.709698 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 14677
E20260928 14:43:07.721477 14774 webserver.cc:533] Webserver: sslize failed:
E20260928 14:43:07.721541 14774 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-28T14:43:07Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (215 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-28T14:43:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:43:07Z Disabled control of system clock
I20260928 14:43:07.756202 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:40591
--webserver_interface=127.9.33.126
--webserver_port=0
--builtin_ntp_servers=127.9.33.84:35537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:40591
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790606561545179-9349-0/testchainca.cert with env {}
W20260928 14:43:07.840758 14778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:07.840972 14778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:07.840996 14778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:07.842176 14778 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:43:07.842219 14778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:07.842232 14778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:07.842247 14778 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:43:07.842293 14778 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260928 14:43:07.842326 14778 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260928 14:43:07.842355 14778 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260928 14:43:07.843976 14778 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:35537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:40591
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:40591
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790606561545179-9349-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14778
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:07.844220 14778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:07.844445 14778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:07.849416 14784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:07.849542 14778 server_base.cc:1034] running on GCE node
W20260928 14:43:07.849426 14783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:07.849438 14786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:07.849908 14778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:07.850236 14778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:07.851434 14778 hybrid_clock.cc:648] HybridClock initialized: now 1790606587851411 us; error 40 us; skew 500 ppm
I20260928 14:43:07.852962 14778 webserver.cc:460] Webserver started at http://127.9.33.126:37301/ using document root <none> and password file <none>
I20260928 14:43:07.853230 14778 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:07.853325 14778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:07.853473 14778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:07.854575 14778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "947fbcd010b341c19ae73f3ccd40596a"
format_stamp: "Formatted at 2026-09-28 14:43:07 on dist-test-slave-pv0w"
server_key: "1dd945fe0de85a2a5473fe254774aac2"
server_key_iv: "a27abd12a0658edfb6bc2453e66279d7"
server_key_version: "encryptionkey@0"
I20260928 14:43:07.854957 14778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "947fbcd010b341c19ae73f3ccd40596a"
format_stamp: "Formatted at 2026-09-28 14:43:07 on dist-test-slave-pv0w"
server_key: "1dd945fe0de85a2a5473fe254774aac2"
server_key_iv: "a27abd12a0658edfb6bc2453e66279d7"
server_key_version: "encryptionkey@0"
I20260928 14:43:07.856477 14778 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:43:07.857324 14792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:07.857517 14778 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:07.857622 14778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "947fbcd010b341c19ae73f3ccd40596a"
format_stamp: "Formatted at 2026-09-28 14:43:07 on dist-test-slave-pv0w"
server_key: "1dd945fe0de85a2a5473fe254774aac2"
server_key_iv: "a27abd12a0658edfb6bc2453e66279d7"
server_key_version: "encryptionkey@0"
I20260928 14:43:07.857699 14778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:07.877514 14778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:07.877841 14778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:07.877997 14778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:07.882004 14778 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:40591
I20260928 14:43:07.882048 14844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:40591 every 8 connection(s)
I20260928 14:43:07.882432 14778 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:43:07.883431 14845 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:07.885844 14845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 947fbcd010b341c19ae73f3ccd40596a: Bootstrap starting.
I20260928 14:43:07.886404 14845 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 947fbcd010b341c19ae73f3ccd40596a: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:07.886641 14845 log.cc:824] T 00000000000000000000000000000000 P 947fbcd010b341c19ae73f3ccd40596a: Log is configured to *not* fsync() on all Append() calls
I20260928 14:43:07.887316 14845 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 947fbcd010b341c19ae73f3ccd40596a: No bootstrap required, opened a new log
I20260928 14:43:07.889019 14845 raft_consensus.cc:348] T 00000000000000000000000000000000 P 947fbcd010b341c19ae73f3ccd40596a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "947fbcd010b341c19ae73f3ccd40596a" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40591 } }
I20260928 14:43:07.889164 14845 raft_consensus.cc:374] T 00000000000000000000000000000000 P 947fbcd010b341c19ae73f3ccd40596a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:07.889189 14845 raft_consensus.cc:729] T 00000000000000000000000000000000 P 947fbcd010b341c19ae73f3ccd40596a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 947fbcd010b341c19ae73f3ccd40596a, State: Initialized, Role: FOLLOWER
I20260928 14:43:07.889329 14845 consensus_queue.cc:260] T 00000000000000000000000000000000 P 947fbcd010b341c19ae73f3ccd40596a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "947fbcd010b341c19ae73f3ccd40596a" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40591 } }
I20260928 14:43:07.889413 14845 raft_consensus.cc:388] T 00000000000000000000000000000000 P 947fbcd010b341c19ae73f3ccd40596a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:43:07.889436 14845 raft_consensus.cc:482] T 00000000000000000000000000000000 P 947fbcd010b341c19ae73f3ccd40596a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:43:07.889465 14845 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 947fbcd010b341c19ae73f3ccd40596a [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:07.889961 14845 raft_consensus.cc:504] T 00000000000000000000000000000000 P 947fbcd010b341c19ae73f3ccd40596a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "947fbcd010b341c19ae73f3ccd40596a" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40591 } }
I20260928 14:43:07.890056 14845 leader_election.cc:302] T 00000000000000000000000000000000 P 947fbcd010b341c19ae73f3ccd40596a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 947fbcd010b341c19ae73f3ccd40596a; no voters:
I20260928 14:43:07.890208 14845 leader_election.cc:288] T 00000000000000000000000000000000 P 947fbcd010b341c19ae73f3ccd40596a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:43:07.890239 14848 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 947fbcd010b341c19ae73f3ccd40596a [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:43:07.890348 14848 raft_consensus.cc:686] T 00000000000000000000000000000000 P 947fbcd010b341c19ae73f3ccd40596a [term 1 LEADER]: Becoming Leader. State: Replica: 947fbcd010b341c19ae73f3ccd40596a, State: Running, Role: LEADER
I20260928 14:43:07.890482 14845 sys_catalog.cc:564] T 00000000000000000000000000000000 P 947fbcd010b341c19ae73f3ccd40596a [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:43:07.890512 14848 consensus_queue.cc:237] T 00000000000000000000000000000000 P 947fbcd010b341c19ae73f3ccd40596a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "947fbcd010b341c19ae73f3ccd40596a" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40591 } }
I20260928 14:43:07.890904 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 14778
I20260928 14:43:07.891021 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:43:07.891211 9349 external_mini_cluster.cc:1428] Setting key 37f36fd427c270007e59d40f6d5e80e8
I20260928 14:43:07.891346 14849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 947fbcd010b341c19ae73f3ccd40596a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "947fbcd010b341c19ae73f3ccd40596a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "947fbcd010b341c19ae73f3ccd40596a" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40591 } } }
I20260928 14:43:07.891431 14849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 947fbcd010b341c19ae73f3ccd40596a [sys.catalog]: This master's current role is: LEADER
I20260928 14:43:07.891628 14850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 947fbcd010b341c19ae73f3ccd40596a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 947fbcd010b341c19ae73f3ccd40596a. Latest consensus state: current_term: 1 leader_uuid: "947fbcd010b341c19ae73f3ccd40596a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "947fbcd010b341c19ae73f3ccd40596a" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40591 } } }
I20260928 14:43:07.891739 14850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 947fbcd010b341c19ae73f3ccd40596a [sys.catalog]: This master's current role is: LEADER
I20260928 14:43:07.891676 14859 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:43:07.892351 14859 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:43:07.893989 14859 catalog_manager.cc:1347] Generated new cluster ID: 3ee9d6866bb04be4b237e60dbfb128fd
I20260928 14:43:07.894039 14859 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:43:07.898597 14867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:07.892067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59468 (local address 127.9.33.126:40591)
0928 14:43:07.892289 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:07.892298 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:07.892331 (+ 33us) server_negotiation.cc:407] Connection header received
0928 14:43:07.892453 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:07.892460 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:07.892533 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:07.892679 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:07.892910 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:07.894168 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:07.895255 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:07.895885 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:07.895992 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:07.896703 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:07.896708 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:07.896708 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:43:07.896921 (+ 213us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:07.896962 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:43:07.896975 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:07.897038 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:07.898294 (+ 1256us) server_negotiation.cc:299] Negotiation successful
0928 14:43:07.898380 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":82,"threads_started":1}
I20260928 14:43:07.899417 9349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:43:07.923938 14859 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:43:07.924697 14859 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:43:07.932086 14859 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 947fbcd010b341c19ae73f3ccd40596a: Generated new TSK 0
I20260928 14:43:07.932314 14859 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:43:07.993618 14867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:07.945723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59472 (local address 127.9.33.126:40591)
0928 14:43:07.945794 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:07.945796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:07.945953 (+ 157us) server_negotiation.cc:407] Connection header received
0928 14:43:07.945993 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:07.945994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:07.946006 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:07.946084 (+ 78us) server_negotiation.cc:226] Negotiated authn=JWT
0928 14:43:07.946249 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:07.946727 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:07.947876 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:07.948127 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:07.948186 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:07.948386 (+ 200us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0928 14:43:07.993399 (+ 45013us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0928 14:43:07.993560 (+ 161us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59472: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":42}
W20260928 14:43:07.993712 14867 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59472: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260928 14:43:07.993916 14874 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:07.945642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:40591 (local address 127.0.0.1:59472)
0928 14:43:07.945832 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:07.945844 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:07.945918 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:07.946105 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:07.946106 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:07.946108 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0928 14:43:07.946174 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:07.946177 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:07.946808 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:07.946809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:07.947793 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:07.947796 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:07.948213 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:07.948214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:07.948275 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:07.948281 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0928 14:43:07.993559 (+ 45278us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0928 14:43:07.993708 (+ 149us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.33.126:40591: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":109,"threads_started":1}
I20260928 14:43:07.994251 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 14778
2026-09-28T14:43:07Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (264 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (90 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (90 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:43:08 dist-test-slave-pv0w krb5kdc[14886](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:43:08 dist-test-slave-pv0w krb5kdc[14886](info): set up 2 sockets
Sep 28 14:43:08 dist-test-slave-pv0w krb5kdc[14886](info): commencing operation
krb5kdc: starting...
W20260928 14:43:09.216838 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:43:09 dist-test-slave-pv0w krb5kdc[14886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:43:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:43:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.33.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
I20260928 14:43:09.342680 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:43169
--webserver_interface=127.9.33.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:34821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:43169
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:43:09.424309 14899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:09.424574 14899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:09.424613 14899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:09.425817 14899 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:43:09.425892 14899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:09.425916 14899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:09.425937 14899 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:43:09.425966 14899 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:43:09.427484 14899 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:34821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:43169
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:43169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14899
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:09.427801 14899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:09.428047 14899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:09.433115 14904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:09.433219 14899 server_base.cc:1034] running on GCE node
W20260928 14:43:09.433106 14907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:09.433104 14905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:09.433612 14899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:09.433939 14899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:09.435098 14899 hybrid_clock.cc:648] HybridClock initialized: now 1790606589435097 us; error 43 us; skew 500 ppm
Sep 28 14:43:09 dist-test-slave-pv0w krb5kdc[14886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606589, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:09.437641 14899 init.cc:377] Logged in from keytab as kudu/127.9.33.126@KRBTEST.COM (short username kudu)
I20260928 14:43:09.438140 14899 webserver.cc:460] Webserver started at http://127.9.33.126:36261/ using document root <none> and password file <none>
I20260928 14:43:09.438381 14899 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:09.438459 14899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:09.438614 14899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:09.439770 14899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "1b9391d246234e889b77aba83ab12bf3"
format_stamp: "Formatted at 2026-09-28 14:43:09 on dist-test-slave-pv0w"
server_key: "f3017bcf15ec23d1f75d1956ab9a6673"
server_key_iv: "123c0ee4b02be021cae673124aa707ed"
server_key_version: "encryptionkey@0"
I20260928 14:43:09.440166 14899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "1b9391d246234e889b77aba83ab12bf3"
format_stamp: "Formatted at 2026-09-28 14:43:09 on dist-test-slave-pv0w"
server_key: "f3017bcf15ec23d1f75d1956ab9a6673"
server_key_iv: "123c0ee4b02be021cae673124aa707ed"
server_key_version: "encryptionkey@0"
I20260928 14:43:09.441598 14899 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:09.442432 14914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:09.442617 14899 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:09.442718 14899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "1b9391d246234e889b77aba83ab12bf3"
format_stamp: "Formatted at 2026-09-28 14:43:09 on dist-test-slave-pv0w"
server_key: "f3017bcf15ec23d1f75d1956ab9a6673"
server_key_iv: "123c0ee4b02be021cae673124aa707ed"
server_key_version: "encryptionkey@0"
I20260928 14:43:09.442808 14899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:09.458815 14899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:09.461509 14899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:09.461635 14899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:09.466301 14899 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:43169
I20260928 14:43:09.466334 14966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:43169 every 8 connection(s)
I20260928 14:43:09.466737 14899 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:43:09.467686 14967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:09.467829 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 14899
I20260928 14:43:09.467939 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:43:09.468125 9349 external_mini_cluster.cc:1428] Setting key d92b51e53fc609fbdd77337c81b04c59
I20260928 14:43:09.471153 14967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b9391d246234e889b77aba83ab12bf3: Bootstrap starting.
I20260928 14:43:09.471859 14967 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b9391d246234e889b77aba83ab12bf3: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:09.472149 14967 log.cc:824] T 00000000000000000000000000000000 P 1b9391d246234e889b77aba83ab12bf3: Log is configured to *not* fsync() on all Append() calls
I20260928 14:43:09.472832 14967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b9391d246234e889b77aba83ab12bf3: No bootstrap required, opened a new log
I20260928 14:43:09.474639 14967 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b9391d246234e889b77aba83ab12bf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b9391d246234e889b77aba83ab12bf3" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 43169 } }
I20260928 14:43:09.474809 14967 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b9391d246234e889b77aba83ab12bf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:09.474856 14967 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b9391d246234e889b77aba83ab12bf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b9391d246234e889b77aba83ab12bf3, State: Initialized, Role: FOLLOWER
I20260928 14:43:09.474992 14967 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b9391d246234e889b77aba83ab12bf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b9391d246234e889b77aba83ab12bf3" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 43169 } }
Sep 28 14:43:09 dist-test-slave-pv0w krb5kdc[14886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:43:09.475091 14967 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b9391d246234e889b77aba83ab12bf3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:43:09.475136 14967 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b9391d246234e889b77aba83ab12bf3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:43:09.475186 14967 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b9391d246234e889b77aba83ab12bf3 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:09.475713 14967 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b9391d246234e889b77aba83ab12bf3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b9391d246234e889b77aba83ab12bf3" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 43169 } }
I20260928 14:43:09.475868 14967 leader_election.cc:302] T 00000000000000000000000000000000 P 1b9391d246234e889b77aba83ab12bf3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b9391d246234e889b77aba83ab12bf3; no voters:
I20260928 14:43:09.476037 14967 leader_election.cc:288] T 00000000000000000000000000000000 P 1b9391d246234e889b77aba83ab12bf3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:43:09.476096 14972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b9391d246234e889b77aba83ab12bf3 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:43:09.476261 14972 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b9391d246234e889b77aba83ab12bf3 [term 1 LEADER]: Becoming Leader. State: Replica: 1b9391d246234e889b77aba83ab12bf3, State: Running, Role: LEADER
I20260928 14:43:09.476392 14967 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b9391d246234e889b77aba83ab12bf3 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:43:09.476418 14972 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b9391d246234e889b77aba83ab12bf3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b9391d246234e889b77aba83ab12bf3" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 43169 } }
I20260928 14:43:09.476892 14974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b9391d246234e889b77aba83ab12bf3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b9391d246234e889b77aba83ab12bf3. Latest consensus state: current_term: 1 leader_uuid: "1b9391d246234e889b77aba83ab12bf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b9391d246234e889b77aba83ab12bf3" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 43169 } } }
I20260928 14:43:09.477164 14974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b9391d246234e889b77aba83ab12bf3 [sys.catalog]: This master's current role is: LEADER
I20260928 14:43:09.477072 14973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b9391d246234e889b77aba83ab12bf3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b9391d246234e889b77aba83ab12bf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b9391d246234e889b77aba83ab12bf3" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 43169 } } }
I20260928 14:43:09.477406 14973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b9391d246234e889b77aba83ab12bf3 [sys.catalog]: This master's current role is: LEADER
I20260928 14:43:09.477578 14981 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:43:09.478185 14981 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:43:09.479771 14970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:09.468669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40830 (local address 127.9.33.126:43169)
0928 14:43:09.468857 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:09.468868 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:09.468906 (+ 38us) server_negotiation.cc:407] Connection header received
0928 14:43:09.469430 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:09.469439 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:09.469504 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:09.469652 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:09.470259 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:09.471640 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:09.472540 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:09.473042 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:09.473172 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:09.475603 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:09.475607 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:09.475607 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:09.475617 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:09.477780 (+ 2163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:09.478192 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:09.478195 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:09.478195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:09.478306 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:09.478550 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:09.478551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:09.478552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:09.478879 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:09.479008 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:09.479411 (+ 403us) server_negotiation.cc:299] Negotiation successful
0928 14:43:09.479553 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":87,"threads_started":1}
I20260928 14:43:09.479801 14981 catalog_manager.cc:1347] Generated new cluster ID: 354143961476443eb67697771dae169a
I20260928 14:43:09.479836 14981 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:43:09.491007 14981 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:43:09.491575 14981 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:43:09.495688 14981 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b9391d246234e889b77aba83ab12bf3: Generated new TSK 0
I20260928 14:43:09.495909 14981 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.33.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab.
Entry for principal kudu/127.9.33.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab.
I20260928 14:43:09.531623 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.65:0
--local_ip_for_outbound_sockets=127.9.33.65
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:43169
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:34821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:43:09.621737 14993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:09.621982 14993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:09.622025 14993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:09.623209 14993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:09.623286 14993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:09.623334 14993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.65
I20260928 14:43:09.624931 14993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:34821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:43169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.65
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:09.625207 14993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:09.625460 14993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:09.630826 14999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:09.630785 14998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:09.630977 14993 server_base.cc:1034] running on GCE node
W20260928 14:43:09.630805 15001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:09.631219 14993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:09.631493 14993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:09.632673 14993 hybrid_clock.cc:648] HybridClock initialized: now 1790606589632638 us; error 50 us; skew 500 ppm
Sep 28 14:43:09 dist-test-slave-pv0w krb5kdc[14886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606589, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:09.635242 14993 init.cc:377] Logged in from keytab as kudu/127.9.33.65@KRBTEST.COM (short username kudu)
I20260928 14:43:09.635833 14993 webserver.cc:460] Webserver started at http://127.9.33.65:38925/ using document root <none> and password file <none>
I20260928 14:43:09.636029 14993 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:09.636070 14993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:09.636164 14993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:09.637418 14993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-0/data/instance:
uuid: "4466a4c61c1a4c57a0954294160eb0e9"
format_stamp: "Formatted at 2026-09-28 14:43:09 on dist-test-slave-pv0w"
server_key: "cf0290905b3f38b5872b3f1e6618c0ec"
server_key_iv: "0f75d2173742ac2f079dfbdeeeeacff9"
server_key_version: "encryptionkey@0"
I20260928 14:43:09.637763 14993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance:
uuid: "4466a4c61c1a4c57a0954294160eb0e9"
format_stamp: "Formatted at 2026-09-28 14:43:09 on dist-test-slave-pv0w"
server_key: "cf0290905b3f38b5872b3f1e6618c0ec"
server_key_iv: "0f75d2173742ac2f079dfbdeeeeacff9"
server_key_version: "encryptionkey@0"
I20260928 14:43:09.639232 14993 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:43:09.640167 15008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:09.640386 14993 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:09.640477 14993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-0/wal
uuid: "4466a4c61c1a4c57a0954294160eb0e9"
format_stamp: "Formatted at 2026-09-28 14:43:09 on dist-test-slave-pv0w"
server_key: "cf0290905b3f38b5872b3f1e6618c0ec"
server_key_iv: "0f75d2173742ac2f079dfbdeeeeacff9"
server_key_version: "encryptionkey@0"
I20260928 14:43:09.640535 14993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:09.668720 14993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:09.671211 14993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:09.671335 14993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:09.671535 14993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:43:09.671914 14993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:43:09.671957 14993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:09.672009 14993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:43:09.672044 14993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:09.678583 14993 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.65:45613
I20260928 14:43:09.678627 15121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.65:45613 every 8 connection(s)
I20260928 14:43:09.679037 14993 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
Sep 28 14:43:09 dist-test-slave-pv0w krb5kdc[14886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606589, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:43:09.687868 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 14993
I20260928 14:43:09.688023 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance
I20260928 14:43:09.688221 9349 external_mini_cluster.cc:1428] Setting key e528baba7115129fad0115344c32eac6
I20260928 14:43:09.689144 14970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:09.679856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:57297 (local address 127.9.33.126:43169)
0928 14:43:09.679923 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:09.679924 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:09.680135 (+ 211us) server_negotiation.cc:407] Connection header received
0928 14:43:09.681153 (+ 1018us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:09.681154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:09.681166 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:09.681224 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:09.681991 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:09.682503 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:09.683560 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:09.683865 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:09.683922 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:09.686159 (+ 2237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:09.686161 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:09.686161 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:09.686169 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:09.687403 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:09.687992 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:09.687993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:09.687994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:09.688062 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:09.688400 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:09.688401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:09.688402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:09.688557 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:09.688625 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:09.688957 (+ 332us) server_negotiation.cc:299] Negotiation successful
0928 14:43:09.689054 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 14:43:09.689347 15124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:09.679864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:43169 (local address 127.9.33.65:57297)
0928 14:43:09.680024 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:09.680062 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:09.681045 (+ 983us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:09.681335 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:09.681339 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:09.681636 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:09.681846 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:09.681851 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:09.682593 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:09.682594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:09.683471 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:09.683474 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:09.683948 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:09.683949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:09.684067 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:09.684553 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:09.684562 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:09.686050 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:09.687478 (+ 1428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:09.687485 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:09.687487 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:09.687860 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:09.688127 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:09.688128 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:09.688128 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:09.688327 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:09.688632 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:09.688640 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:09.688800 (+ 160us) client_negotiation.cc:769] Sending connection context
0928 14:43:09.688913 (+ 113us) client_negotiation.cc:240] Negotiation successful
0928 14:43:09.689089 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":86,"threads_started":1}
I20260928 14:43:09.689837 15122 heartbeater.cc:347] Connected to a master server at 127.9.33.126:43169
I20260928 14:43:09.689956 15122 heartbeater.cc:464] Registering TS with master...
I20260928 14:43:09.690145 15122 heartbeater.cc:511] Master 127.9.33.126:43169 requested a full tablet report, sending...
I20260928 14:43:09.690788 14931 ts_manager.cc:194] Registered new tserver with Master: 4466a4c61c1a4c57a0954294160eb0e9 (127.9.33.65:45613)
I20260928 14:43:09.691421 14931 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.65@KRBTEST.COM'} at 127.9.33.65:57297
WARNING: no policy specified for kudu/127.9.33.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab.
Entry for principal kudu/127.9.33.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab.
I20260928 14:43:09.715162 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.66:0
--local_ip_for_outbound_sockets=127.9.33.66
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:43169
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:34821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:43:09.798892 15127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:09.799163 15127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:09.799189 15127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:09.800700 15127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:09.800753 15127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:09.800783 15127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.66
I20260928 14:43:09.802448 15127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:34821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:43169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.66
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:09.802733 15127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:09.802995 15127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:09.808490 15135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:09.808470 15132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:09.808540 15133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:09.808765 15127 server_base.cc:1034] running on GCE node
I20260928 14:43:09.809010 15127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:09.809232 15127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:09.810384 15127 hybrid_clock.cc:648] HybridClock initialized: now 1790606589810368 us; error 29 us; skew 500 ppm
Sep 28 14:43:09 dist-test-slave-pv0w krb5kdc[14886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606589, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:09.813424 15127 init.cc:377] Logged in from keytab as kudu/127.9.33.66@KRBTEST.COM (short username kudu)
I20260928 14:43:09.813998 15127 webserver.cc:460] Webserver started at http://127.9.33.66:35315/ using document root <none> and password file <none>
I20260928 14:43:09.814252 15127 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:09.814328 15127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:09.814497 15127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:09.815774 15127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-1/data/instance:
uuid: "29b4d428d551426e8e80abbff60212ca"
format_stamp: "Formatted at 2026-09-28 14:43:09 on dist-test-slave-pv0w"
server_key: "94e469a36bf66ac9e9a7f2319edf8679"
server_key_iv: "03642d77cd72492acffcfa2824a694e9"
server_key_version: "encryptionkey@0"
I20260928 14:43:09.816201 15127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance:
uuid: "29b4d428d551426e8e80abbff60212ca"
format_stamp: "Formatted at 2026-09-28 14:43:09 on dist-test-slave-pv0w"
server_key: "94e469a36bf66ac9e9a7f2319edf8679"
server_key_iv: "03642d77cd72492acffcfa2824a694e9"
server_key_version: "encryptionkey@0"
I20260928 14:43:09.817663 15127 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:43:09.818467 15142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:09.818641 15127 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:43:09.818754 15127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-1/wal
uuid: "29b4d428d551426e8e80abbff60212ca"
format_stamp: "Formatted at 2026-09-28 14:43:09 on dist-test-slave-pv0w"
server_key: "94e469a36bf66ac9e9a7f2319edf8679"
server_key_iv: "03642d77cd72492acffcfa2824a694e9"
server_key_version: "encryptionkey@0"
I20260928 14:43:09.818845 15127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:09.836882 15127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:09.839679 15127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:09.839900 15127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:09.840211 15127 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:43:09.840579 15127 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:43:09.840641 15127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:09.840680 15127 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:43:09.840711 15127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:09.847556 15127 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.66:42007
I20260928 14:43:09.847595 15255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.66:42007 every 8 connection(s)
I20260928 14:43:09.848038 15127 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
I20260928 14:43:09.850591 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 15127
I20260928 14:43:09.850693 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance
I20260928 14:43:09.850905 9349 external_mini_cluster.cc:1428] Setting key bece438941dc40e3c38dd81bb4f5ac53
Sep 28 14:43:09 dist-test-slave-pv0w krb5kdc[14886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606589, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.33.67@KRBTEST.COM; defaulting to no policy
I20260928 14:43:09.860421 14970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:09.848936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:39467 (local address 127.9.33.126:43169)
0928 14:43:09.849012 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:09.849015 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:09.849231 (+ 216us) server_negotiation.cc:407] Connection header received
0928 14:43:09.850210 (+ 979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:09.850211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:09.850224 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:09.850282 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:09.851278 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:09.852070 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:09.853573 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:09.853770 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:09.853881 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:09.856559 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:09.856561 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:09.856561 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:09.856571 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:09.858421 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:09.858978 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:09.858979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:09.858980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:09.859048 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:09.859336 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:09.859337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:09.859338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:09.859502 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:09.859587 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:09.860200 (+ 613us) server_negotiation.cc:299] Negotiation successful
0928 14:43:09.860304 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 14:43:09.860599 15258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:09.848937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:43169 (local address 127.9.33.66:39467)
0928 14:43:09.849094 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:09.849131 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:09.850094 (+ 963us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:09.850364 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:09.850368 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:09.850783 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:09.851137 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:09.851146 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:09.852170 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:09.852172 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:09.853465 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:09.853468 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:09.853889 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:09.853890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:09.854071 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:09.854659 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:09.854669 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:09.856448 (+ 1779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:09.858552 (+ 2104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:09.858562 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:09.858564 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:09.858848 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:09.859132 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:09.859133 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:09.859133 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:09.859259 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:09.859587 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:09.859593 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:09.860024 (+ 431us) client_negotiation.cc:769] Sending connection context
0928 14:43:09.860160 (+ 136us) client_negotiation.cc:240] Negotiation successful
0928 14:43:09.860358 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":79,"threads_started":1}
I20260928 14:43:09.861194 15256 heartbeater.cc:347] Connected to a master server at 127.9.33.126:43169
I20260928 14:43:09.861308 15256 heartbeater.cc:464] Registering TS with master...
I20260928 14:43:09.861509 15256 heartbeater.cc:511] Master 127.9.33.126:43169 requested a full tablet report, sending...
I20260928 14:43:09.862004 14931 ts_manager.cc:194] Registered new tserver with Master: 29b4d428d551426e8e80abbff60212ca (127.9.33.66:42007)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.67@KRBTEST.COM" created.
I20260928 14:43:09.862598 14931 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.66@KRBTEST.COM'} at 127.9.33.66:39467
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab.
Entry for principal kudu/127.9.33.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab.
I20260928 14:43:09.878969 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.67:0
--local_ip_for_outbound_sockets=127.9.33.67
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:43169
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:34821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:43:09.960336 15261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:09.960546 15261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:09.960570 15261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:09.961752 15261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:09.961797 15261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:09.961825 15261 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.67
I20260928 14:43:09.963413 15261 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:34821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:43169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.67
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:09.963680 15261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:09.963935 15261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:09.969210 15266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:09.969241 15261 server_base.cc:1034] running on GCE node
W20260928 14:43:09.969198 15269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:09.969177 15267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:09.969645 15261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:09.969861 15261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:09.971014 15261 hybrid_clock.cc:648] HybridClock initialized: now 1790606589970999 us; error 28 us; skew 500 ppm
Sep 28 14:43:09 dist-test-slave-pv0w krb5kdc[14886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606589, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:09.973977 15261 init.cc:377] Logged in from keytab as kudu/127.9.33.67@KRBTEST.COM (short username kudu)
I20260928 14:43:09.974462 15261 webserver.cc:460] Webserver started at http://127.9.33.67:40955/ using document root <none> and password file <none>
I20260928 14:43:09.974704 15261 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:09.974769 15261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:09.974864 15261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:09.975970 15261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-2/data/instance:
uuid: "5213c32acfde4a5dbfca5d2344794b06"
format_stamp: "Formatted at 2026-09-28 14:43:09 on dist-test-slave-pv0w"
server_key: "330cbaff2f6ebcb0bf68650553385865"
server_key_iv: "606cfbc73ce15ba87e4d359156c762f9"
server_key_version: "encryptionkey@0"
I20260928 14:43:09.976338 15261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance:
uuid: "5213c32acfde4a5dbfca5d2344794b06"
format_stamp: "Formatted at 2026-09-28 14:43:09 on dist-test-slave-pv0w"
server_key: "330cbaff2f6ebcb0bf68650553385865"
server_key_iv: "606cfbc73ce15ba87e4d359156c762f9"
server_key_version: "encryptionkey@0"
I20260928 14:43:09.977937 15261 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:43:09.978734 15276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:09.978930 15261 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:09.979010 15261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-2/wal
uuid: "5213c32acfde4a5dbfca5d2344794b06"
format_stamp: "Formatted at 2026-09-28 14:43:09 on dist-test-slave-pv0w"
server_key: "330cbaff2f6ebcb0bf68650553385865"
server_key_iv: "606cfbc73ce15ba87e4d359156c762f9"
server_key_version: "encryptionkey@0"
I20260928 14:43:09.979102 15261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:09.991183 15261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:09.993438 15261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:09.993597 15261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:09.993897 15261 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:43:09.994318 15261 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:43:09.994354 15261 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:09.994377 15261 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:43:09.994423 15261 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:10.000754 15261 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.67:45995
I20260928 14:43:10.000851 15389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.67:45995 every 8 connection(s)
I20260928 14:43:10.001286 15261 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
I20260928 14:43:10.004266 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 15261
I20260928 14:43:10.004387 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance
I20260928 14:43:10.004573 9349 external_mini_cluster.cc:1428] Setting key 192690d50544969a95424f2f7912724f
Sep 28 14:43:10 dist-test-slave-pv0w krb5kdc[14886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606589, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:43:10.012585 14970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:10.002130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:46901 (local address 127.9.33.126:43169)
0928 14:43:10.002199 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:10.002201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:10.002409 (+ 208us) server_negotiation.cc:407] Connection header received
0928 14:43:10.003351 (+ 942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:10.003352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:10.003364 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:10.003421 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:10.004188 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:10.004893 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:10.006042 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:10.006239 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:10.006347 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:10.008618 (+ 2271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:10.008620 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:10.008620 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:10.008629 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:10.010560 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:10.011243 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:10.011245 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:10.011245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:10.011315 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:10.011663 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:10.011664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:10.011665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:10.011847 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:10.011931 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:10.012339 (+ 408us) server_negotiation.cc:299] Negotiation successful
0928 14:43:10.012452 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260928 14:43:10.012794 15392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:10.002122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:43169 (local address 127.9.33.67:46901)
0928 14:43:10.002271 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:10.002306 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:10.003245 (+ 939us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:10.003489 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:10.003492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:10.003829 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:10.004062 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:10.004066 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:10.004996 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:10.004997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:10.005936 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:10.005939 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:10.006357 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:10.006365 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:10.006495 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:10.006993 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:10.007001 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:10.008519 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:10.010657 (+ 2138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:10.010665 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:10.010667 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:10.011107 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:10.011409 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:10.011410 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:10.011411 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:10.011567 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:10.011952 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:10.011957 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:10.012163 (+ 206us) client_negotiation.cc:769] Sending connection context
0928 14:43:10.012295 (+ 132us) client_negotiation.cc:240] Negotiation successful
0928 14:43:10.012526 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":85,"threads_started":1}
I20260928 14:43:10.013628 15390 heartbeater.cc:347] Connected to a master server at 127.9.33.126:43169
I20260928 14:43:10.013743 15390 heartbeater.cc:464] Registering TS with master...
I20260928 14:43:10.013950 15390 heartbeater.cc:511] Master 127.9.33.126:43169 requested a full tablet report, sending...
I20260928 14:43:10.014430 14931 ts_manager.cc:194] Registered new tserver with Master: 5213c32acfde4a5dbfca5d2344794b06 (127.9.33.67:45995)
I20260928 14:43:10.014997 14931 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.67@KRBTEST.COM'} at 127.9.33.67:46901
I20260928 14:43:10.016364 9349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/krb5kdc/test-admin.keytab.
Sep 28 14:43:10 dist-test-slave-pv0w krb5kdc[14886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:10.033977 9349 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260928 14:43:10.035879 15397 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:10.034893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47774 (local address 127.9.33.65:45613)
0928 14:43:10.035072 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:10.035076 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:10.035089 (+ 13us) server_negotiation.cc:407] Connection header received
0928 14:43:10.035135 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:10.035137 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:10.035152 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:10.035228 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:10.035719 (+ 491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47774: BlockingRecv error: recv got EOF from 127.0.0.1:47774 (error 108)
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":95,"threads_started":1}
W20260928 14:43:10.035897 9349 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/krb5kdc/krb5cc' of type FILE
Sep 28 14:43:10 dist-test-slave-pv0w krb5kdc[14886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:10.037108 9349 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 28 14:43:10 dist-test-slave-pv0w krb5kdc[14886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:10.038009 9349 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 28 14:43:10 dist-test-slave-pv0w krb5kdc[14886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.65@KRBTEST.COM
I20260928 14:43:10.047169 15397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:10.038553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47786 (local address 127.9.33.65:45613)
0928 14:43:10.038624 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:10.038626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:10.038733 (+ 107us) server_negotiation.cc:407] Connection header received
0928 14:43:10.038823 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:10.038824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:10.038844 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:10.038964 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:10.039381 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:10.040197 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:10.040994 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:10.041343 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:10.041431 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:10.043874 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:10.043927 (+ 53us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:10.043928 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:10.043938 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:10.045862 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:10.046256 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:10.046260 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:10.046261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:10.046327 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:10.046569 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:10.046571 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:10.046571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:10.046751 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:10.046897 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:10.046936 (+ 39us) server_negotiation.cc:299] Negotiation successful
0928 14:43:10.047007 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260928 14:43:10.049147 15397 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:10.048556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47796 (local address 127.9.33.65:45613)
0928 14:43:10.048596 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:10.048597 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:10.048623 (+ 26us) server_negotiation.cc:407] Connection header received
0928 14:43:10.048700 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:10.048701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:10.048712 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:10.048785 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:10.049097 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47796: BlockingRecv error: recv got EOF from 127.0.0.1:47796 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260928 14:43:10.049220 9349 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/krb5kdc/krb5cc' of type FILE
Sep 28 14:43:10 dist-test-slave-pv0w krb5kdc[14886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:10.050209 9349 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 28 14:43:10 dist-test-slave-pv0w krb5kdc[14886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:10.051069 9349 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 28 14:43:10 dist-test-slave-pv0w krb5kdc[14886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.65@KRBTEST.COM
I20260928 14:43:10.058240 15397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:10.051635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47806 (local address 127.9.33.65:45613)
0928 14:43:10.051692 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:10.051694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:10.051752 (+ 58us) server_negotiation.cc:407] Connection header received
0928 14:43:10.051853 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:10.051854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:10.051864 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:10.051976 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:10.052343 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:10.052848 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:10.053544 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:10.053757 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:10.053844 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:10.055869 (+ 2025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:10.055871 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:10.055871 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:10.055886 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:10.057080 (+ 1194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:10.057455 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:10.057457 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:10.057457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:10.057516 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:10.057726 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:10.057727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:10.057727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:10.057834 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:10.057951 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:10.058048 (+ 97us) server_negotiation.cc:299] Negotiation successful
0928 14:43:10.058131 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260928 14:43:10.060035 15397 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:10.059339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47810 (local address 127.9.33.65:45613)
0928 14:43:10.059393 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:10.059395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:10.059484 (+ 89us) server_negotiation.cc:407] Connection header received
0928 14:43:10.059538 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:10.059539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:10.059549 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:10.059603 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:10.059985 (+ 382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47810: BlockingRecv error: recv got EOF from 127.0.0.1:47810 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260928 14:43:10.060073 9349 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790606561545179-9349-0/krb5kdc/krb5cc' of type FILE
Sep 28 14:43:10 dist-test-slave-pv0w krb5kdc[14886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:10.061020 9349 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 28 14:43:10 dist-test-slave-pv0w krb5kdc[14886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:10.061870 9349 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 28 14:43:10 dist-test-slave-pv0w krb5kdc[14886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.65@KRBTEST.COM
I20260928 14:43:10.069082 15397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:10.062375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47818 (local address 127.9.33.65:45613)
0928 14:43:10.062430 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:10.062432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:10.062493 (+ 61us) server_negotiation.cc:407] Connection header received
0928 14:43:10.062565 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:10.062567 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:10.062575 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:10.062623 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:10.062989 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:10.063463 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:10.064225 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:10.064440 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:10.064516 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:10.066550 (+ 2034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:10.066552 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:10.066552 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:10.066559 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:10.067953 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:10.068316 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:10.068318 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:10.068318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:10.068377 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:10.068583 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:10.068584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:10.068584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:10.068690 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:10.068792 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:10.068906 (+ 114us) server_negotiation.cc:299] Negotiation successful
0928 14:43:10.068981 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260928 14:43:10.069803 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 14993
I20260928 14:43:10.074601 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 15127
I20260928 14:43:10.079345 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 15261
I20260928 14:43:10.083623 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 14899
2026-09-28T14:43:10Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1909 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:43:10 dist-test-slave-pv0w krb5kdc[15410](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:43:10 dist-test-slave-pv0w krb5kdc[15410](info): set up 2 sockets
Sep 28 14:43:10 dist-test-slave-pv0w krb5kdc[15410](info): commencing operation
krb5kdc: starting...
W20260928 14:43:11.124790 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:43:11 dist-test-slave-pv0w krb5kdc[15410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:43:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:43:11Z Disabled control of system clock
WARNING: no policy specified for oryx/127.9.33.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.33.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.33.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.126.keytab.
Entry for principal oryx/127.9.33.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.126.keytab.
I20260928 14:43:11.250959 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:38209
--webserver_interface=127.9.33.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.126.keytab
--principal=oryx/127.9.33.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:41001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:38209
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:43:11.337036 15423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:11.337286 15423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:11.337316 15423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:11.338639 15423 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:43:11.338707 15423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:11.338723 15423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:11.338737 15423 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:43:11.338748 15423 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:43:11.340550 15423 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:41001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:38209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.126.keytab
--principal=oryx/127.9.33.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:38209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:11.340875 15423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:11.341154 15423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:11.346905 15429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:11.346912 15428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:11.346935 15431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:11.347258 15423 server_base.cc:1034] running on GCE node
I20260928 14:43:11.347523 15423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:11.347903 15423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:11.349088 15423 hybrid_clock.cc:648] HybridClock initialized: now 1790606591349055 us; error 51 us; skew 500 ppm
Sep 28 14:43:11 dist-test-slave-pv0w krb5kdc[15410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606591, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:11.352125 15423 init.cc:377] Logged in from keytab as oryx/127.9.33.126@KRBTEST.COM (short username oryx)
I20260928 14:43:11.352674 15423 webserver.cc:460] Webserver started at http://127.9.33.126:45173/ using document root <none> and password file <none>
I20260928 14:43:11.352937 15423 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:11.353012 15423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:11.353201 15423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:11.354355 15423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "4cfad45886d34600a02df0ae697f86b2"
format_stamp: "Formatted at 2026-09-28 14:43:11 on dist-test-slave-pv0w"
server_key: "4969a94231b7a363ae498a3464378b2f"
server_key_iv: "06c407efc3bea2c20f2f53c7dabe0bd0"
server_key_version: "encryptionkey@0"
I20260928 14:43:11.354823 15423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "4cfad45886d34600a02df0ae697f86b2"
format_stamp: "Formatted at 2026-09-28 14:43:11 on dist-test-slave-pv0w"
server_key: "4969a94231b7a363ae498a3464378b2f"
server_key_iv: "06c407efc3bea2c20f2f53c7dabe0bd0"
server_key_version: "encryptionkey@0"
I20260928 14:43:11.356452 15423 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:43:11.357365 15438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:11.357587 15423 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:43:11.357703 15423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "4cfad45886d34600a02df0ae697f86b2"
format_stamp: "Formatted at 2026-09-28 14:43:11 on dist-test-slave-pv0w"
server_key: "4969a94231b7a363ae498a3464378b2f"
server_key_iv: "06c407efc3bea2c20f2f53c7dabe0bd0"
server_key_version: "encryptionkey@0"
I20260928 14:43:11.357781 15423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:11.370551 15423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:11.373049 15423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:11.373251 15423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:11.377801 15423 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:38209
I20260928 14:43:11.377832 15490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:38209 every 8 connection(s)
I20260928 14:43:11.378207 15423 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:43:11.379205 15491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:11.381686 15491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2: Bootstrap starting.
I20260928 14:43:11.382309 15491 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:11.382624 15491 log.cc:824] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2: Log is configured to *not* fsync() on all Append() calls
I20260928 14:43:11.383522 15491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2: No bootstrap required, opened a new log
I20260928 14:43:11.385222 15491 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } }
I20260928 14:43:11.385376 15491 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:11.385401 15491 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cfad45886d34600a02df0ae697f86b2, State: Initialized, Role: FOLLOWER
I20260928 14:43:11.385573 15491 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } }
I20260928 14:43:11.385648 15491 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:43:11.385671 15491 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:43:11.385742 15491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:11.386499 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 15423
I20260928 14:43:11.386622 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:43:11.386785 15491 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } }
I20260928 14:43:11.386837 9349 external_mini_cluster.cc:1428] Setting key 634383681b9d89498463a01e4e1da105
I20260928 14:43:11.386900 15491 leader_election.cc:302] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4cfad45886d34600a02df0ae697f86b2; no voters:
I20260928 14:43:11.387079 15491 leader_election.cc:288] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:43:11.387135 15494 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:43:11.387334 15494 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [term 1 LEADER]: Becoming Leader. State: Replica: 4cfad45886d34600a02df0ae697f86b2, State: Running, Role: LEADER
I20260928 14:43:11.387378 15491 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:43:11.387513 15494 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } }
I20260928 14:43:11.387969 15495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4cfad45886d34600a02df0ae697f86b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } }
I20260928 14:43:11.388063 15495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [sys.catalog]: This master's current role is: LEADER
I20260928 14:43:11.388221 15496 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4cfad45886d34600a02df0ae697f86b2. Latest consensus state: current_term: 1 leader_uuid: "4cfad45886d34600a02df0ae697f86b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } }
I20260928 14:43:11.388280 15496 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [sys.catalog]: This master's current role is: LEADER
I20260928 14:43:11.388586 15505 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:43:11.389241 15505 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:43:11.390920 15505 catalog_manager.cc:1347] Generated new cluster ID: 063a364e0f4846ca91dc5cd374c69e34
I20260928 14:43:11.390987 15505 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 14:43:11 dist-test-slave-pv0w krb5kdc[15410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM
I20260928 14:43:11.399174 15498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:11.387541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46258 (local address 127.9.33.126:38209)
0928 14:43:11.387739 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:11.387749 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:11.387785 (+ 36us) server_negotiation.cc:407] Connection header received
0928 14:43:11.387910 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:11.387918 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:11.387983 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:11.388709 (+ 726us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:11.388947 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:11.390360 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:11.391440 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:11.392108 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:11.392226 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:11.395167 (+ 2941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:11.395172 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:11.395172 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:11.395187 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:11.397415 (+ 2228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:11.397830 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:11.397834 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:11.397834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:11.397972 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:11.398177 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:11.398178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:11.398179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:11.398522 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:11.398694 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:11.398771 (+ 77us) server_negotiation.cc:299] Negotiation successful
0928 14:43:11.398931 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":63,"threads_started":1}
I20260928 14:43:11.402942 15505 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:43:11.403517 15505 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:43:11.409650 15505 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2: Generated new TSK 0
I20260928 14:43:11.409965 15505 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.9.33.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.33.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.33.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.65.keytab.
Entry for principal oryx/127.9.33.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.65.keytab.
I20260928 14:43:11.442199 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.65:0
--local_ip_for_outbound_sockets=127.9.33.65
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:38209
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.65.keytab
--principal=oryx/127.9.33.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:41001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:43:11.543912 15517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:11.544139 15517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:11.544167 15517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:11.545449 15517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:11.545501 15517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:11.545531 15517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.65
I20260928 14:43:11.547163 15517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:41001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.65.keytab
--principal=oryx/127.9.33.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:38209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.65
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:11.547454 15517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:11.547763 15517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:11.553965 15522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:11.554302 15517 server_base.cc:1034] running on GCE node
W20260928 14:43:11.554443 15523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:11.554489 15525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:11.554777 15517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:11.555018 15517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:11.556173 15517 hybrid_clock.cc:648] HybridClock initialized: now 1790606591556159 us; error 29 us; skew 500 ppm
Sep 28 14:43:11 dist-test-slave-pv0w krb5kdc[15410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606591, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:11.559055 15517 init.cc:377] Logged in from keytab as oryx/127.9.33.65@KRBTEST.COM (short username oryx)
I20260928 14:43:11.559567 15517 webserver.cc:460] Webserver started at http://127.9.33.65:36105/ using document root <none> and password file <none>
I20260928 14:43:11.559851 15517 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:11.559906 15517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:11.560007 15517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:11.561242 15517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-0/data/instance:
uuid: "dc15cdcdf4b145588845d6ce64504136"
format_stamp: "Formatted at 2026-09-28 14:43:11 on dist-test-slave-pv0w"
server_key: "6dde13c35c8dbbdd70fd8a26245fd8a7"
server_key_iv: "186301989e4c4a284793d2c6f12533ff"
server_key_version: "encryptionkey@0"
I20260928 14:43:11.561618 15517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance:
uuid: "dc15cdcdf4b145588845d6ce64504136"
format_stamp: "Formatted at 2026-09-28 14:43:11 on dist-test-slave-pv0w"
server_key: "6dde13c35c8dbbdd70fd8a26245fd8a7"
server_key_iv: "186301989e4c4a284793d2c6f12533ff"
server_key_version: "encryptionkey@0"
I20260928 14:43:11.563099 15517 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:43:11.563968 15532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:11.564191 15517 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:11.564291 15517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-0/wal
uuid: "dc15cdcdf4b145588845d6ce64504136"
format_stamp: "Formatted at 2026-09-28 14:43:11 on dist-test-slave-pv0w"
server_key: "6dde13c35c8dbbdd70fd8a26245fd8a7"
server_key_iv: "186301989e4c4a284793d2c6f12533ff"
server_key_version: "encryptionkey@0"
I20260928 14:43:11.564371 15517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:11.582839 15517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:11.585942 15517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:11.586100 15517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:11.586385 15517 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:43:11.586741 15517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:43:11.586798 15517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:11.586843 15517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:43:11.586871 15517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:11.593505 15517 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.65:39055
I20260928 14:43:11.593545 15645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.65:39055 every 8 connection(s)
I20260928 14:43:11.593917 15517 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
I20260928 14:43:11.598546 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 15517
I20260928 14:43:11.598635 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance
I20260928 14:43:11.598860 9349 external_mini_cluster.cc:1428] Setting key 47f439e976a791f75ad7a00c0e75f28d
Sep 28 14:43:11 dist-test-slave-pv0w krb5kdc[15410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606591, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM
I20260928 14:43:11.604182 15498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:11.594874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:58697 (local address 127.9.33.126:38209)
0928 14:43:11.594952 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:11.594953 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:11.595185 (+ 232us) server_negotiation.cc:407] Connection header received
0928 14:43:11.595766 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:11.595768 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:11.595781 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:11.595828 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:11.596872 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:11.597394 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:11.598452 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:11.598723 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:11.598798 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:11.601172 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:11.601175 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:11.601175 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:11.601183 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:11.602624 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:11.603076 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:11.603077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:11.603078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:11.603141 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:11.603409 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:11.603410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:11.603411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:11.603547 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:11.603621 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:11.603998 (+ 377us) server_negotiation.cc:299] Negotiation successful
0928 14:43:11.604085 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260928 14:43:11.604406 15648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:11.594869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38209 (local address 127.9.33.65:58697)
0928 14:43:11.595050 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:11.595089 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:11.595633 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:11.595905 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:11.595910 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:11.596537 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:11.596748 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:11.596754 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:11.597494 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:11.597495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:11.598351 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:11.598352 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:11.598819 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:11.598820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:11.599008 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:11.599578 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:11.599586 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:11.601071 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:11.602729 (+ 1658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:11.602737 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:11.602740 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:11.602995 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:11.603211 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:11.603212 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:11.603212 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:11.603335 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:11.603625 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:11.603632 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:11.603853 (+ 221us) client_negotiation.cc:769] Sending connection context
0928 14:43:11.603973 (+ 120us) client_negotiation.cc:240] Negotiation successful
0928 14:43:11.604149 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":89,"threads_started":1}
I20260928 14:43:11.604911 15646 heartbeater.cc:347] Connected to a master server at 127.9.33.126:38209
I20260928 14:43:11.605013 15646 heartbeater.cc:464] Registering TS with master...
I20260928 14:43:11.605225 15646 heartbeater.cc:511] Master 127.9.33.126:38209 requested a full tablet report, sending...
I20260928 14:43:11.605810 15455 ts_manager.cc:194] Registered new tserver with Master: dc15cdcdf4b145588845d6ce64504136 (127.9.33.65:39055)
I20260928 14:43:11.606587 15455 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.33.65@KRBTEST.COM'} at 127.9.33.65:58697
WARNING: no policy specified for oryx/127.9.33.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.33.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.33.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.66.keytab.
Entry for principal oryx/127.9.33.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.66.keytab.
I20260928 14:43:11.627832 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.66:0
--local_ip_for_outbound_sockets=127.9.33.66
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:38209
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.66.keytab
--principal=oryx/127.9.33.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:41001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:43:11.712396 15651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:11.712621 15651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:11.712648 15651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:11.713811 15651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:11.713853 15651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:11.713881 15651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.66
I20260928 14:43:11.715390 15651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:41001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.66.keytab
--principal=oryx/127.9.33.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:38209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.66
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:11.715600 15651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:11.715822 15651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:11.721205 15657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:11.721184 15659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:11.721175 15656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:11.721402 15651 server_base.cc:1034] running on GCE node
I20260928 14:43:11.721675 15651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:11.721995 15651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:11.723169 15651 hybrid_clock.cc:648] HybridClock initialized: now 1790606591723155 us; error 29 us; skew 500 ppm
Sep 28 14:43:11 dist-test-slave-pv0w krb5kdc[15410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606591, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:11.725908 15651 init.cc:377] Logged in from keytab as oryx/127.9.33.66@KRBTEST.COM (short username oryx)
I20260928 14:43:11.726475 15651 webserver.cc:460] Webserver started at http://127.9.33.66:35435/ using document root <none> and password file <none>
I20260928 14:43:11.726732 15651 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:11.726805 15651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:11.726960 15651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:11.728184 15651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-1/data/instance:
uuid: "b5fdff97460e49f5acef24e0105aab26"
format_stamp: "Formatted at 2026-09-28 14:43:11 on dist-test-slave-pv0w"
server_key: "a420c9ef74f63fee892ea11925ae97ae"
server_key_iv: "f015b9587c00e6d1c6d93091a35946f7"
server_key_version: "encryptionkey@0"
I20260928 14:43:11.728570 15651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance:
uuid: "b5fdff97460e49f5acef24e0105aab26"
format_stamp: "Formatted at 2026-09-28 14:43:11 on dist-test-slave-pv0w"
server_key: "a420c9ef74f63fee892ea11925ae97ae"
server_key_iv: "f015b9587c00e6d1c6d93091a35946f7"
server_key_version: "encryptionkey@0"
I20260928 14:43:11.730059 15651 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:43:11.730937 15666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:11.731153 15651 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:11.731267 15651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-1/wal
uuid: "b5fdff97460e49f5acef24e0105aab26"
format_stamp: "Formatted at 2026-09-28 14:43:11 on dist-test-slave-pv0w"
server_key: "a420c9ef74f63fee892ea11925ae97ae"
server_key_iv: "f015b9587c00e6d1c6d93091a35946f7"
server_key_version: "encryptionkey@0"
I20260928 14:43:11.731350 15651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:11.754110 15651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:11.756310 15651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:11.756503 15651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:11.756760 15651 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:43:11.757119 15651 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:43:11.757180 15651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:11.757228 15651 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:43:11.757259 15651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:11.764312 15651 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.66:37471
I20260928 14:43:11.764397 15779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.66:37471 every 8 connection(s)
I20260928 14:43:11.764791 15651 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
Sep 28 14:43:11 dist-test-slave-pv0w krb5kdc[15410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606591, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM
I20260928 14:43:11.773561 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 15651
I20260928 14:43:11.773689 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance
I20260928 14:43:11.773907 9349 external_mini_cluster.cc:1428] Setting key 8e0ae3c55edc15c4a3048b330f84bd84
I20260928 14:43:11.775803 15498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:11.765826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:60831 (local address 127.9.33.126:38209)
0928 14:43:11.765918 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:11.765921 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:11.766134 (+ 213us) server_negotiation.cc:407] Connection header received
0928 14:43:11.766640 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:11.766642 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:11.766655 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:11.766728 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:11.767974 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:11.768644 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:11.769921 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:11.770144 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:11.770213 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:11.772623 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:11.772625 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:11.772625 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:11.772633 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:11.774147 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:11.774734 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:11.774735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:11.774736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:11.774813 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:11.775097 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:11.775098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:11.775099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:11.775221 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:11.775297 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:11.775594 (+ 297us) server_negotiation.cc:299] Negotiation successful
0928 14:43:11.775681 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260928 14:43:11.776021 15782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:11.765684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38209 (local address 127.9.33.66:60831)
0928 14:43:11.765980 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:11.766024 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:11.766531 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:11.766831 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:11.766836 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:11.767594 (+ 758us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:11.767835 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:11.767845 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:11.768752 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:11.768753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:11.769801 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:11.769804 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:11.770247 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:11.770248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:11.770377 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:11.770998 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:11.771007 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:11.772524 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:11.774326 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:11.774334 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:11.774336 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:11.774641 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:11.774882 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:11.774883 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:11.774883 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:11.775015 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:11.775288 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:11.775295 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:11.775454 (+ 159us) client_negotiation.cc:769] Sending connection context
0928 14:43:11.775562 (+ 108us) client_negotiation.cc:240] Negotiation successful
0928 14:43:11.775755 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":165,"threads_started":1}
I20260928 14:43:11.776747 15780 heartbeater.cc:347] Connected to a master server at 127.9.33.126:38209
I20260928 14:43:11.776861 15780 heartbeater.cc:464] Registering TS with master...
I20260928 14:43:11.777074 15780 heartbeater.cc:511] Master 127.9.33.126:38209 requested a full tablet report, sending...
I20260928 14:43:11.777491 15455 ts_manager.cc:194] Registered new tserver with Master: b5fdff97460e49f5acef24e0105aab26 (127.9.33.66:37471)
I20260928 14:43:11.778018 15455 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.33.66@KRBTEST.COM'} at 127.9.33.66:60831
WARNING: no policy specified for oryx/127.9.33.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.33.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.33.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.67.keytab.
Entry for principal oryx/127.9.33.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.67.keytab.
I20260928 14:43:11.803431 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.67:0
--local_ip_for_outbound_sockets=127.9.33.67
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:38209
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.67.keytab
--principal=oryx/127.9.33.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:41001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:43:11.892176 15785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:11.892396 15785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:11.892421 15785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:11.893622 15785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:11.893664 15785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:11.893692 15785 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.67
I20260928 14:43:11.895192 15785 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:41001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.67.keytab
--principal=oryx/127.9.33.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:38209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.67
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:11.895385 15785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:11.895570 15785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:11.901062 15793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:11.901114 15785 server_base.cc:1034] running on GCE node
W20260928 14:43:11.901214 15791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:11.901129 15790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:11.901569 15785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:11.901895 15785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:11.903087 15785 hybrid_clock.cc:648] HybridClock initialized: now 1790606591903079 us; error 45 us; skew 500 ppm
Sep 28 14:43:11 dist-test-slave-pv0w krb5kdc[15410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606591, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:11.905882 15785 init.cc:377] Logged in from keytab as oryx/127.9.33.67@KRBTEST.COM (short username oryx)
I20260928 14:43:11.906349 15785 webserver.cc:460] Webserver started at http://127.9.33.67:38031/ using document root <none> and password file <none>
I20260928 14:43:11.906584 15785 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:11.906652 15785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:11.906819 15785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:11.908063 15785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-2/data/instance:
uuid: "71899b6c516945508de4a4770b126e5f"
format_stamp: "Formatted at 2026-09-28 14:43:11 on dist-test-slave-pv0w"
server_key: "c8a7f79f3df858481c9f3df30099e540"
server_key_iv: "f84f062f41d706b68171bf5e06957ffd"
server_key_version: "encryptionkey@0"
I20260928 14:43:11.908440 15785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance:
uuid: "71899b6c516945508de4a4770b126e5f"
format_stamp: "Formatted at 2026-09-28 14:43:11 on dist-test-slave-pv0w"
server_key: "c8a7f79f3df858481c9f3df30099e540"
server_key_iv: "f84f062f41d706b68171bf5e06957ffd"
server_key_version: "encryptionkey@0"
I20260928 14:43:11.909979 15785 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:11.910742 15800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:11.910923 15785 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:43:11.911027 15785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-2/wal
uuid: "71899b6c516945508de4a4770b126e5f"
format_stamp: "Formatted at 2026-09-28 14:43:11 on dist-test-slave-pv0w"
server_key: "c8a7f79f3df858481c9f3df30099e540"
server_key_iv: "f84f062f41d706b68171bf5e06957ffd"
server_key_version: "encryptionkey@0"
I20260928 14:43:11.911105 15785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:11.939839 15785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:11.942268 15785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:11.942458 15785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:11.942714 15785 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:43:11.943063 15785 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:43:11.943120 15785 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:11.943157 15785 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:43:11.943207 15785 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:11.949342 15785 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.67:34405
I20260928 14:43:11.949394 15913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.67:34405 every 8 connection(s)
I20260928 14:43:11.949791 15785 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
Sep 28 14:43:11 dist-test-slave-pv0w krb5kdc[15410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606591, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM
I20260928 14:43:11.959677 15498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:11.950655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:38417 (local address 127.9.33.126:38209)
0928 14:43:11.950736 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:11.950738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:11.950946 (+ 208us) server_negotiation.cc:407] Connection header received
0928 14:43:11.951417 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:11.951419 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:11.951430 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:11.951500 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:11.952586 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:11.953111 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:11.954189 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:11.954376 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:11.954431 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:11.956821 (+ 2390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:11.956823 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:11.956823 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:11.956831 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:11.958295 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:11.958726 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:11.958727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:11.958727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:11.958782 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:11.959068 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:11.959069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:11.959069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:11.959188 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:11.959243 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:11.959517 (+ 274us) server_negotiation.cc:299] Negotiation successful
0928 14:43:11.959585 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260928 14:43:11.959877 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 15785
I20260928 14:43:11.959976 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance
I20260928 14:43:11.960022 15916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:11.950600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38209 (local address 127.9.33.67:38417)
0928 14:43:11.950758 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:11.950802 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:11.951297 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:11.951556 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:11.951560 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:11.952241 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:11.952453 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:11.952459 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:11.953212 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:11.953213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:11.954111 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:11.954114 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:11.954450 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:11.954451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:11.954552 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:11.955110 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:11.955119 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:11.956716 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:11.958392 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:11.958399 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:11.958401 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:11.958644 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:11.958883 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:11.958884 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:11.958884 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:11.958995 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:11.959249 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:11.959253 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:11.959396 (+ 143us) client_negotiation.cc:769] Sending connection context
0928 14:43:11.959483 (+ 87us) client_negotiation.cc:240] Negotiation successful
0928 14:43:11.959614 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":87,"threads_started":1}
I20260928 14:43:11.960191 9349 external_mini_cluster.cc:1428] Setting key e28dddb517d2726236b517d92ab3cf6a
I20260928 14:43:11.960620 15914 heartbeater.cc:347] Connected to a master server at 127.9.33.126:38209
I20260928 14:43:11.960757 15914 heartbeater.cc:464] Registering TS with master...
I20260928 14:43:11.960981 15914 heartbeater.cc:511] Master 127.9.33.126:38209 requested a full tablet report, sending...
I20260928 14:43:11.961452 15454 ts_manager.cc:194] Registered new tserver with Master: 71899b6c516945508de4a4770b126e5f (127.9.33.67:34405)
I20260928 14:43:11.962014 15454 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.33.67@KRBTEST.COM'} at 127.9.33.67:38417
I20260928 14:43:11.972381 9349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.9.33.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.33.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.33.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.125.keytab.
Entry for principal oryx/127.9.33.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.125.keytab.
I20260928 14:43:11.999886 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.125:36321
--webserver_interface=127.9.33.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.125.keytab
--principal=oryx/127.9.33.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:41001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:38209,127.9.33.125:36321
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:43:12.087329 15919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:12.087651 15919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:12.087677 15919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:12.089053 15919 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:43:12.089102 15919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:12.089116 15919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:12.089129 15919 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:43:12.089141 15919 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:43:12.090632 15919 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:41001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:38209,127.9.33.125:36321
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.125.keytab
--principal=oryx/127.9.33.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.125:36321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.33.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
W20260928 14:43:12.090739 15919 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.9.33.126:38209,127.9.33.125:36321'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260928 14:43:12.090864 15919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:12.091040 15919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:12.096129 15924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:12.096127 15925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:12.096123 15927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:12.096496 15919 server_base.cc:1034] running on GCE node
I20260928 14:43:12.096804 15919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:12.097155 15919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:12.098327 15919 hybrid_clock.cc:648] HybridClock initialized: now 1790606592098302 us; error 65 us; skew 500 ppm
Sep 28 14:43:12 dist-test-slave-pv0w krb5kdc[15410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606592, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:12.101047 15919 init.cc:377] Logged in from keytab as oryx/127.9.33.125@KRBTEST.COM (short username oryx)
I20260928 14:43:12.101524 15919 webserver.cc:460] Webserver started at http://127.9.33.125:44745/ using document root <none> and password file <none>
I20260928 14:43:12.101780 15919 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:12.101852 15919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:12.102005 15919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:12.103195 15919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-1/data/instance:
uuid: "39a978772cff40a6882ec08ae1ae2f52"
format_stamp: "Formatted at 2026-09-28 14:43:12 on dist-test-slave-pv0w"
server_key: "93cdaf6bbb25aac4674970954ce94883"
server_key_iv: "58f612262181825360a71d25a68776e1"
server_key_version: "encryptionkey@0"
I20260928 14:43:12.103577 15919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-1/wal/instance:
uuid: "39a978772cff40a6882ec08ae1ae2f52"
format_stamp: "Formatted at 2026-09-28 14:43:12 on dist-test-slave-pv0w"
server_key: "93cdaf6bbb25aac4674970954ce94883"
server_key_iv: "58f612262181825360a71d25a68776e1"
server_key_version: "encryptionkey@0"
I20260928 14:43:12.105049 15919 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:43:12.105810 15934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:12.105988 15919 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:12.106078 15919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-1/wal
uuid: "39a978772cff40a6882ec08ae1ae2f52"
format_stamp: "Formatted at 2026-09-28 14:43:12 on dist-test-slave-pv0w"
server_key: "93cdaf6bbb25aac4674970954ce94883"
server_key_iv: "58f612262181825360a71d25a68776e1"
server_key_version: "encryptionkey@0"
I20260928 14:43:12.106163 15919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:12.120553 15919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:12.122462 15919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:12.122633 15919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:12.126892 15919 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.125:36321
I20260928 14:43:12.126925 15986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.125:36321 every 8 connection(s)
I20260928 14:43:12.127322 15919 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-1/data/info.pb
I20260928 14:43:12.128404 15987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:12.130052 15987 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:43:12.135392 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 15919
Sep 28 14:43:12 dist-test-slave-pv0w krb5kdc[15410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606592, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.125@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM
I20260928 14:43:12.140559 15498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.130861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46270 (local address 127.9.33.126:38209)
0928 14:43:12.131004 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:12.131006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:12.131114 (+ 108us) server_negotiation.cc:407] Connection header received
0928 14:43:12.131590 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:12.131591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:12.131601 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:12.131669 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:12.132377 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.132897 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.133928 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.134201 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.134260 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.137100 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:12.137102 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:12.137103 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:12.137114 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:12.138921 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.139430 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.139431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.139432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.139497 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.139810 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.139811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.139812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.139979 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:12.140060 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:12.140373 (+ 313us) server_negotiation.cc:299] Negotiation successful
0928 14:43:12.140464 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260928 14:43:12.140810 15989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.130820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38209 (local address 127.0.0.1:46270)
0928 14:43:12.130998 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:12.131040 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:12.131500 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:12.131781 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:12.131785 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:12.132053 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:12.132262 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.132266 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.132999 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.133000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.133824 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.133827 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.134276 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.134277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.134384 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.135006 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:12.135015 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:12.136955 (+ 1940us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:12.139004 (+ 2049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.139009 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.139011 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.139333 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.139567 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.139568 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.139568 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.139697 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.140063 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:12.140070 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:12.140224 (+ 154us) client_negotiation.cc:769] Sending connection context
0928 14:43:12.140330 (+ 106us) client_negotiation.cc:240] Negotiation successful
0928 14:43:12.140496 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":102,"threads_started":1}
Sep 28 14:43:12 dist-test-slave-pv0w krb5kdc[15410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.33.125@KRBTEST.COM
I20260928 14:43:12.141531 15987 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36321 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:43:12.145524 15991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.135988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47044 (local address 127.9.33.125:36321)
0928 14:43:12.136301 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:12.136306 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:12.136326 (+ 20us) server_negotiation.cc:407] Connection header received
0928 14:43:12.136345 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:12.136346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:12.136385 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:12.136473 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:12.137226 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.137935 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.138708 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.139083 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.139154 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.142070 (+ 2916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:12.142079 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:12.142079 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:12.142087 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:12.143889 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.144300 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.144304 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.144304 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.144383 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.144616 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.144617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.144617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.144915 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:12.145093 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:12.145275 (+ 182us) server_negotiation.cc:299] Negotiation successful
0928 14:43:12.145398 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":83,"threads_started":1}
Sep 28 14:43:12 dist-test-slave-pv0w krb5kdc[15410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606592, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.125@KRBTEST.COM for oryx/127.9.33.125@KRBTEST.COM
I20260928 14:43:12.150048 15989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.141803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.125:36321 (local address 127.0.0.1:47054)
0928 14:43:12.141853 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:12.141922 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:12.141964 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:12.142261 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:12.142262 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:12.142384 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:12.142432 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.142434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.143487 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.143488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.144221 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.144223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.144734 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.144735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.144819 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.145339 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:12.145344 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:12.146654 (+ 1310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:12.148953 (+ 2299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.148954 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.148955 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.149157 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.149362 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.149362 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.149362 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.149416 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.149609 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:12.149610 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:12.149647 (+ 37us) client_negotiation.cc:769] Sending connection context
0928 14:43:12.149746 (+ 99us) client_negotiation.cc:240] Negotiation successful
0928 14:43:12.149842 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260928 14:43:12.150051 15992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.141853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47054 (local address 127.9.33.125:36321)
0928 14:43:12.142105 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:12.142127 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:12.142136 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:43:12.142154 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:12.142155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:12.142166 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:12.142253 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:12.142517 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.143402 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.144324 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.144644 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.144697 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.146726 (+ 2029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:12.146727 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:12.146727 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:12.146733 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:12.148848 (+ 2115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.149235 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.149236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.149236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.149292 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.149458 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.149459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.149459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.149557 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:12.149603 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:12.149780 (+ 177us) server_negotiation.cc:299] Negotiation successful
0928 14:43:12.149904 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":228,"thread_start_us":138,"threads_started":1}
I20260928 14:43:12.151613 15987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52: Bootstrap starting.
I20260928 14:43:12.152320 15987 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:12.152657 15987 log.cc:824] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52: Log is configured to *not* fsync() on all Append() calls
I20260928 14:43:12.153602 15987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52: No bootstrap required, opened a new log
I20260928 14:43:12.155131 15987 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36321 } }
I20260928 14:43:12.155264 15987 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:12.155287 15987 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39a978772cff40a6882ec08ae1ae2f52, State: Initialized, Role: FOLLOWER
I20260928 14:43:12.155445 15987 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36321 } }
I20260928 14:43:12.155848 15987 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:43:12.155812 15994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36321 } } }
I20260928 14:43:12.155920 15994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [sys.catalog]: This master's current role is: FOLLOWER
W20260928 14:43:12.157493 16005 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:43:12.157555 16005 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:43:12.159358 15919 master_runner.cc:428] Detected that this master 39a978772cff40a6882ec08ae1ae2f52 is joining an existing cluster
I20260928 14:43:12.159435 15919 master_runner.cc:432] Initiating AddMaster RPC to add 127.9.33.125:36321
I20260928 14:43:12.168314 15498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.160689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46274 (local address 127.9.33.126:38209)
0928 14:43:12.160744 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:12.160747 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:12.161048 (+ 301us) server_negotiation.cc:407] Connection header received
0928 14:43:12.161113 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:12.161115 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:12.161126 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:12.161176 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:12.161569 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.162211 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.163118 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.163326 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.163407 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.164655 (+ 1248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:12.164656 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:12.164657 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:12.164665 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:12.166797 (+ 2132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.167294 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.167295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.167296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.167364 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.167663 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.167664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.167664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.167851 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:12.167927 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:12.168128 (+ 201us) server_negotiation.cc:299] Negotiation successful
0928 14:43:12.168213 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260928 14:43:12.168334 16012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.160672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38209 (local address 127.0.0.1:46274)
0928 14:43:12.160987 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:12.160993 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:12.161063 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:12.161177 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:12.161178 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:12.161428 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:12.161490 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.161492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.162308 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.162309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.163016 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.163017 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.163485 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.163486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.163545 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.164086 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:12.164090 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:12.164562 (+ 472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:12.166895 (+ 2333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.166896 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.166896 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.167198 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.167450 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.167451 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.167451 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.167519 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.167937 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:12.167938 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:12.167990 (+ 52us) client_negotiation.cc:769] Sending connection context
0928 14:43:12.168083 (+ 93us) client_negotiation.cc:240] Negotiation successful
0928 14:43:12.168198 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"mutex_wait_us":42,"thread_start_us":112,"threads_started":1}
I20260928 14:43:12.169414 16011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.160656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.125:36321 (local address 127.0.0.1:47062)
0928 14:43:12.160834 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:12.160844 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:12.160903 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:12.161069 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:12.161070 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:12.161362 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:12.161441 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.161451 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.162425 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.162425 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.163287 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.163289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.164088 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.164089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.164161 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.164628 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:12.164632 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:12.165001 (+ 369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:12.167821 (+ 2820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.167822 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.167822 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.168061 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.168245 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.168246 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.168246 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.168302 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.169048 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:12.169049 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:12.169087 (+ 38us) client_negotiation.cc:769] Sending connection context
0928 14:43:12.169169 (+ 82us) client_negotiation.cc:240] Negotiation successful
0928 14:43:12.169284 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":121,"threads_started":1}
I20260928 14:43:12.169483 15992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.160875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47062 (local address 127.9.33.125:36321)
0928 14:43:12.160919 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:12.160921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:12.160929 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:43:12.160954 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:12.160955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:12.160969 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:12.161028 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:12.161540 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.162354 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.163373 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.163665 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.163766 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.165114 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:12.165116 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:12.165116 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:12.165123 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:12.167706 (+ 2583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.168133 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.168134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.168134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.168185 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.168831 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.168832 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.168832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.168956 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:12.169003 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:12.169221 (+ 218us) server_negotiation.cc:299] Negotiation successful
0928 14:43:12.169342 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":23}
Sep 28 14:43:12 dist-test-slave-pv0w krb5kdc[15410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606591, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.126@KRBTEST.COM for oryx/127.9.33.125@KRBTEST.COM
I20260928 14:43:12.179698 15992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.171026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47072 (local address 127.9.33.125:36321)
0928 14:43:12.171093 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:12.171095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:12.171960 (+ 865us) server_negotiation.cc:407] Connection header received
0928 14:43:12.172041 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:12.172041 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:12.172050 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:12.172096 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:12.172637 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.173127 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.174293 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.174519 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.174602 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.176934 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:12.176936 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:12.176936 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:12.176944 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:12.178373 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.178820 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.178822 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.178822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.178889 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.179112 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.179113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.179113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.179235 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:12.179313 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:12.179505 (+ 192us) server_negotiation.cc:299] Negotiation successful
0928 14:43:12.179592 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260928 14:43:12.179724 16013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.170974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.125:36321 (local address 127.0.0.1:47072)
0928 14:43:12.171834 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:12.171883 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:12.171964 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:12.172101 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:12.172105 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:12.172476 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:12.172542 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.172544 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.173212 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.173213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.174188 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.174191 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.174604 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.174605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.174696 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.175218 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:12.175224 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:12.176826 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:12.178486 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.178494 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.178497 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.178722 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.178959 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.178961 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.178961 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.179028 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.179305 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:12.179313 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:12.179371 (+ 58us) client_negotiation.cc:769] Sending connection context
0928 14:43:12.179478 (+ 107us) client_negotiation.cc:240] Negotiation successful
0928 14:43:12.179609 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":829,"thread_start_us":93,"threads_started":1}
I20260928 14:43:12.180547 15454 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.9.33.125:36321: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52"
member_type: NON_VOTER
last_known_addr {
host: "127.9.33.125"
port: 36321
}
attrs {
promote: true
}
}
dest_uuid: "4cfad45886d34600a02df0ae697f86b2"
cas_config_opid_index: -1
I20260928 14:43:12.180898 15454 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: NON_VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: true } }
I20260928 14:43:12.181427 16015 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 39a978772cff40a6882ec08ae1ae2f52 (127.9.33.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: NON_VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: true } } }
I20260928 14:43:12.181792 15961 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:12.182411 16015 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.9.33.125:36321
I20260928 14:43:12.182629 15919 master.cc:537] Master@127.9.33.125:36321 shutting down...
I20260928 14:43:12.183259 15961 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [term 1 FOLLOWER]: Refusing update from remote peer 4cfad45886d34600a02df0ae697f86b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260928 14:43:12.183754 16016 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: NON_VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:43:12.184087 16014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4cfad45886d34600a02df0ae697f86b2" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: NON_VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: true } } }
I20260928 14:43:12.184206 16014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [sys.catalog]: This master's current role is: LEADER
W20260928 14:43:12.185232 15441 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260928 14:43:12.185617 15441 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 -> Peer 39a978772cff40a6882ec08ae1ae2f52 (127.9.33.125:36321): Couldn't send request to peer 39a978772cff40a6882ec08ae1ae2f52. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260928 14:43:12.185892 15919 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:43:12.185999 15919 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:43:12.186054 15919 tablet_replica.cc:330] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52: stopping tablet replica
I20260928 14:43:12.188628 15919 master.cc:552] Master@127.9.33.125:36321 shutdown complete.
I20260928 14:43:12.188694 15919 master_runner.cc:305] Clearing existing system tablet
I20260928 14:43:12.188833 15919 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:12.188880 15919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:12.189515 15919 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:43:12.190039 16020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:12.190166 15919 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:12.190223 15919 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:12.190245 15919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-1/wal
uuid: "39a978772cff40a6882ec08ae1ae2f52"
format_stamp: "Formatted at 2026-09-28 14:43:12 on dist-test-slave-pv0w"
server_key: "93cdaf6bbb25aac4674970954ce94883"
server_key_iv: "58f612262181825360a71d25a68776e1"
server_key_version: "encryptionkey@0"
I20260928 14:43:12.190821 15919 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:43:12.192183 15919 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260928 14:43:12.192240 15919 log.cc:1196] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260928 14:43:12.192471 15919 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52: Deleting consensus metadata
I20260928 14:43:12.192615 15919 master_runner.cc:315] Copying system tablet from 127.9.33.126:38209
I20260928 14:43:12.193216 15919 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52: tablet copy: Beginning tablet copy session from remote peer at address 127.9.33.126:38209
I20260928 14:43:12.199874 16026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.193547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38209 (local address 127.0.0.1:46276)
0928 14:43:12.193725 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:12.193737 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:12.193810 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:12.193983 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:12.193984 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:12.194227 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:12.194280 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.194282 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.195015 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.195016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.195598 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.195600 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.196047 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.196048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.196118 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.196612 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:12.196618 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:12.197037 (+ 419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:12.198695 (+ 1658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.198696 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.198696 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.198914 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.199164 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.199165 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.199165 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.199233 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.199490 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:12.199491 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:12.199541 (+ 50us) client_negotiation.cc:769] Sending connection context
0928 14:43:12.199619 (+ 78us) client_negotiation.cc:240] Negotiation successful
0928 14:43:12.199753 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":77,"threads_started":1}
I20260928 14:43:12.199906 15498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.193733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46276 (local address 127.9.33.126:38209)
0928 14:43:12.193785 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:12.193787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:12.193797 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:43:12.193890 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:12.193891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:12.193900 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:12.193979 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:12.194367 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.194925 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.195698 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.195961 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.196046 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.197131 (+ 1085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:12.197133 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:12.197133 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:12.197139 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:12.198609 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.199021 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.199022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.199023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.199084 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.199300 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.199301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.199301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.199427 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:12.199482 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:12.199635 (+ 153us) server_negotiation.cc:299] Negotiation successful
0928 14:43:12.199701 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260928 14:43:12.200239 15475 tablet_copy_service.cc:140] P 4cfad45886d34600a02df0ae697f86b2: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 39a978772cff40a6882ec08ae1ae2f52 ({username='oryx', principal='oryx/127.9.33.125@KRBTEST.COM'} at 127.0.0.1:46276)
I20260928 14:43:12.200346 15475 tablet_copy_service.cc:161] P 4cfad45886d34600a02df0ae697f86b2: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 39a978772cff40a6882ec08ae1ae2f52 at {username='oryx', principal='oryx/127.9.33.125@KRBTEST.COM'} at 127.0.0.1:46276: session id = 39a978772cff40a6882ec08ae1ae2f52-00000000000000000000000000000000
I20260928 14:43:12.201223 15475 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2: Tablet Copy: opened 0 blocks and 1 log segments
I20260928 14:43:12.202008 15919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260928 14:43:12.203104 15919 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52: tablet copy: Starting download of 0 data blocks...
I20260928 14:43:12.203280 15919 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52: tablet copy: Starting download of 1 WAL segments...
I20260928 14:43:12.204397 15919 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260928 14:43:12.205359 15475 tablet_copy_service.cc:342] P 4cfad45886d34600a02df0ae697f86b2: Request end of tablet copy session 39a978772cff40a6882ec08ae1ae2f52-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.9.33.125@KRBTEST.COM'} at 127.0.0.1:46276
I20260928 14:43:12.205454 15475 tablet_copy_service.cc:433] P 4cfad45886d34600a02df0ae697f86b2: ending tablet copy session 39a978772cff40a6882ec08ae1ae2f52-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 39a978772cff40a6882ec08ae1ae2f52
I20260928 14:43:12.208261 15919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:12.209619 16028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:12.209766 15919 server_base.cc:1034] running on GCE node
W20260928 14:43:12.209673 16030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:12.209955 16027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:12.210122 15919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:12.210291 15919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:12.211442 15919 hybrid_clock.cc:648] HybridClock initialized: now 1790606592211433 us; error 24 us; skew 500 ppm
Sep 28 14:43:12 dist-test-slave-pv0w krb5kdc[15410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606592, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:12.212931 15919 init.cc:377] Logged in from keytab as oryx/127.9.33.125@KRBTEST.COM (short username oryx)
I20260928 14:43:12.213234 15919 webserver.cc:460] Webserver started at http://127.9.33.125:38157/ using document root <none> and password file <none>
I20260928 14:43:12.213407 15919 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:12.213447 15919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:12.214097 15919 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:43:12.214658 16037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:12.214792 15919 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:12.214846 15919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-1/wal
uuid: "39a978772cff40a6882ec08ae1ae2f52"
format_stamp: "Formatted at 2026-09-28 14:43:12 on dist-test-slave-pv0w"
server_key: "93cdaf6bbb25aac4674970954ce94883"
server_key_iv: "58f612262181825360a71d25a68776e1"
server_key_version: "encryptionkey@0"
I20260928 14:43:12.214910 15919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:12.222620 15919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:12.224480 15919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:12.227664 15919 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.125:36321
I20260928 14:43:12.227685 16089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.125:36321 every 8 connection(s)
I20260928 14:43:12.228102 15919 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-1/data/info.pb
I20260928 14:43:12.228929 16090 sys_catalog.cc:263] Verifying existing consensus state
I20260928 14:43:12.229439 16090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52: Bootstrap starting.
I20260928 14:43:12.232576 16090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 14:43:12.232815 16090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52: Bootstrap complete.
I20260928 14:43:12.233017 16090 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: NON_VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: true } }
I20260928 14:43:12.233114 16090 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 39a978772cff40a6882ec08ae1ae2f52, State: Initialized, Role: LEARNER
I20260928 14:43:12.233157 16090 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: NON_VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: true } }
I20260928 14:43:12.233325 16090 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:43:12.234645 16093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: NON_VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: true } } }
I20260928 14:43:12.234835 16093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [sys.catalog]: This master's current role is: LEARNER
I20260928 14:43:12.235229 16104 catalog_manager.cc:1259] Loaded cluster ID: 063a364e0f4846ca91dc5cd374c69e34
I20260928 14:43:12.235311 16104 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52: loading cluster ID for follower catalog manager: success
I20260928 14:43:12.236176 16104 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52: acquiring CA information for follower catalog manager: success
I20260928 14:43:12.236464 16104 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 14:43:12.255355 16105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.247373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47086 (local address 127.9.33.125:36321)
0928 14:43:12.247525 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:12.247528 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:12.247537 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:43:12.247564 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:12.247564 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:12.247573 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:12.247654 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:12.248483 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.249216 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.250317 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.250541 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.250612 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.252216 (+ 1604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:12.252218 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:12.252218 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:12.252225 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:12.253922 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.254350 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.254351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.254351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.254424 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.254650 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.254651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.254651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.254787 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:12.254874 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:12.255078 (+ 204us) server_negotiation.cc:299] Negotiation successful
0928 14:43:12.255166 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":98,"threads_started":1}
WARNING: no policy specified for oryx/127.9.33.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.33.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.33.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.124.keytab.
Entry for principal oryx/127.9.33.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.124.keytab.
I20260928 14:43:12.282954 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.124:40843
--webserver_interface=127.9.33.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.124.keytab
--principal=oryx/127.9.33.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:41001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:38209,127.9.33.125:36321,127.9.33.124:40843
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:43:12.368407 16108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:12.368633 16108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:12.368659 16108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:12.369918 16108 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:43:12.369973 16108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:12.369988 16108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:12.370002 16108 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:43:12.370014 16108 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:43:12.371645 16108 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:41001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:38209,127.9.33.125:36321,127.9.33.124:40843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.124.keytab
--principal=oryx/127.9.33.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.124:40843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.33.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16108
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:12.371961 16108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:12.372263 16108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:12.377364 16116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:12.377329 16114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:12.377323 16113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:12.377562 16108 server_base.cc:1034] running on GCE node
I20260928 14:43:12.377812 16108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:12.378122 16108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:12.379354 16108 hybrid_clock.cc:648] HybridClock initialized: now 1790606592379340 us; error 31 us; skew 500 ppm
Sep 28 14:43:12 dist-test-slave-pv0w krb5kdc[15410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606592, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:12.381881 16108 init.cc:377] Logged in from keytab as oryx/127.9.33.124@KRBTEST.COM (short username oryx)
I20260928 14:43:12.382354 16108 webserver.cc:460] Webserver started at http://127.9.33.124:38643/ using document root <none> and password file <none>
I20260928 14:43:12.382550 16108 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:12.382591 16108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:12.382680 16108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:12.383710 16108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-2/data/instance:
uuid: "ba6dfd46bca84cc5a9551e578c07703d"
format_stamp: "Formatted at 2026-09-28 14:43:12 on dist-test-slave-pv0w"
server_key: "be2fa3866172817b5830f05e5a258c53"
server_key_iv: "86e1bc4d1d56ebdcd2a7a1aa5e658053"
server_key_version: "encryptionkey@0"
I20260928 14:43:12.384092 16108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-2/wal/instance:
uuid: "ba6dfd46bca84cc5a9551e578c07703d"
format_stamp: "Formatted at 2026-09-28 14:43:12 on dist-test-slave-pv0w"
server_key: "be2fa3866172817b5830f05e5a258c53"
server_key_iv: "86e1bc4d1d56ebdcd2a7a1aa5e658053"
server_key_version: "encryptionkey@0"
I20260928 14:43:12.385444 16108 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:12.386240 16123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:12.386413 16108 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:12.386497 16108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-2/wal
uuid: "ba6dfd46bca84cc5a9551e578c07703d"
format_stamp: "Formatted at 2026-09-28 14:43:12 on dist-test-slave-pv0w"
server_key: "be2fa3866172817b5830f05e5a258c53"
server_key_iv: "86e1bc4d1d56ebdcd2a7a1aa5e658053"
server_key_version: "encryptionkey@0"
I20260928 14:43:12.386554 16108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:12.401870 16108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:12.404307 16108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:12.404482 16108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:12.408768 16108 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.124:40843
I20260928 14:43:12.408828 16175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.124:40843 every 8 connection(s)
I20260928 14:43:12.409251 16108 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-2/data/info.pb
I20260928 14:43:12.410378 16176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:12.412135 16176 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 14:43:12 dist-test-slave-pv0w krb5kdc[15410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606592, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.124@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM
I20260928 14:43:12.418417 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 16108
I20260928 14:43:12.422254 15498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.413013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46284 (local address 127.9.33.126:38209)
0928 14:43:12.413109 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:12.413112 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:12.413300 (+ 188us) server_negotiation.cc:407] Connection header received
0928 14:43:12.413942 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:12.413943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:12.413957 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:12.414004 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:12.414671 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.415269 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.416452 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.416659 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.416746 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.419102 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:12.419104 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:12.419104 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:12.419113 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:12.420716 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.421168 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.421169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.421170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.421234 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.421516 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.421517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.421518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.421640 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:12.421723 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:12.422067 (+ 344us) server_negotiation.cc:299] Negotiation successful
0928 14:43:12.422164 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260928 14:43:12.422435 16178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.412859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38209 (local address 127.0.0.1:46284)
0928 14:43:12.413134 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:12.413181 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:12.413777 (+ 596us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:12.414084 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:12.414087 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:12.414346 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:12.414547 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.414552 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.415371 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.415372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.416356 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.416359 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.416757 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.416758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.416891 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.417386 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:12.417395 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:12.418983 (+ 1588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:12.420820 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.420827 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.420830 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.421071 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.421324 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.421325 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.421325 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.421444 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.421713 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:12.421720 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:12.421882 (+ 162us) client_negotiation.cc:769] Sending connection context
0928 14:43:12.422024 (+ 142us) client_negotiation.cc:240] Negotiation successful
0928 14:43:12.422189 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"mutex_wait_us":1,"thread_start_us":124,"threads_started":1}
I20260928 14:43:12.423069 16176 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36321 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 14:43:12 dist-test-slave-pv0w krb5kdc[15410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.33.124@KRBTEST.COM
Sep 28 14:43:12 dist-test-slave-pv0w krb5kdc[15410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606592, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.124@KRBTEST.COM for oryx/127.9.33.125@KRBTEST.COM
I20260928 14:43:12.427977 16179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.419012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50986 (local address 127.9.33.124:40843)
0928 14:43:12.419192 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:12.419201 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:12.419215 (+ 14us) server_negotiation.cc:407] Connection header received
0928 14:43:12.419233 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:12.419235 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:12.419245 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:12.419304 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:12.420130 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.420804 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.421534 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.421846 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.421925 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.424717 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:12.424725 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:12.424725 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:12.424733 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:12.426616 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.426997 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.427003 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.427003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.427062 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.427337 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.427337 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.427338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.427480 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:12.427639 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:12.427713 (+ 74us) server_negotiation.cc:299] Negotiation successful
0928 14:43:12.427825 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":119,"threads_started":1}
I20260928 14:43:12.430191 16178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.423344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.125:36321 (local address 127.0.0.1:47102)
0928 14:43:12.423394 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:12.423405 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:12.423441 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:12.423606 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:12.423607 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:12.423830 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:12.423878 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.423879 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.424625 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.424626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.425471 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.425474 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.425859 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.425860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.425932 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.426272 (+ 340us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:12.426276 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:12.427482 (+ 1206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:12.429299 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.429300 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.429301 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.429461 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.429653 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.429653 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.429653 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.429703 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.429974 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:12.429974 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:12.430009 (+ 35us) client_negotiation.cc:769] Sending connection context
0928 14:43:12.430058 (+ 49us) client_negotiation.cc:240] Negotiation successful
0928 14:43:12.430130 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260928 14:43:12.430202 16105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.423430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47102 (local address 127.9.33.125:36321)
0928 14:43:12.423481 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:12.423483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:12.423493 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:43:12.423518 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:12.423519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:12.423531 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:12.423604 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:12.423961 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.424520 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.425539 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.425771 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.425850 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.427574 (+ 1724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:12.427576 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:12.427576 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:12.427584 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:12.429216 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.429532 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.429533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.429534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.429593 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.429755 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.429755 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.429755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.429917 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:12.429969 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:12.430077 (+ 108us) server_negotiation.cc:299] Negotiation successful
0928 14:43:12.430142 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 14:43:12.430650 16176 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.33.124" port: 40843 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 14:43:12 dist-test-slave-pv0w krb5kdc[15410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606592, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.124@KRBTEST.COM for oryx/127.9.33.124@KRBTEST.COM
I20260928 14:43:12.437467 16178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.430967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:40843 (local address 127.0.0.1:51000)
0928 14:43:12.431021 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:12.431033 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:12.431076 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:12.431186 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:12.431187 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:12.431294 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:12.431327 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.431328 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.432095 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.432096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.432569 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.432570 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.432955 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.432956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.432997 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.433589 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:12.433594 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:12.435025 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:12.436467 (+ 1442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.436468 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.436468 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.436624 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.436871 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.436872 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.436872 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.436925 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.437229 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:12.437229 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:12.437265 (+ 36us) client_negotiation.cc:769] Sending connection context
0928 14:43:12.437318 (+ 53us) client_negotiation.cc:240] Negotiation successful
0928 14:43:12.437396 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260928 14:43:12.437620 16179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.431019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51000 (local address 127.9.33.124:40843)
0928 14:43:12.431053 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:12.431054 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:12.431063 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:43:12.431136 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:12.431137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:12.431144 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:12.431183 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:12.431413 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.431990 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.432646 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.432880 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.432938 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.435100 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:12.435101 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:12.435101 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:12.435107 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:12.436361 (+ 1254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.436710 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.436712 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.436712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.436767 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.437030 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.437032 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.437032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.437156 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:12.437226 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:12.437337 (+ 111us) server_negotiation.cc:299] Negotiation successful
0928 14:43:12.437447 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":18}
I20260928 14:43:12.438954 16176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d: Bootstrap starting.
I20260928 14:43:12.439858 16176 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:12.440155 16176 log.cc:824] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d: Log is configured to *not* fsync() on all Append() calls
I20260928 14:43:12.440892 16176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d: No bootstrap required, opened a new log
I20260928 14:43:12.442373 16176 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36321 } } peers { permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 40843 } }
I20260928 14:43:12.442504 16176 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:12.442528 16176 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba6dfd46bca84cc5a9551e578c07703d, State: Initialized, Role: FOLLOWER
I20260928 14:43:12.442658 16176 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36321 } } peers { permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 40843 } }
I20260928 14:43:12.442996 16176 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:43:12.442999 16181 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36321 } } peers { permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 40843 } } }
I20260928 14:43:12.443181 16181 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d [sys.catalog]: This master's current role is: FOLLOWER
W20260928 14:43:12.444870 16192 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:43:12.444936 16192 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:43:12.447659 16108 master_runner.cc:428] Detected that this master ba6dfd46bca84cc5a9551e578c07703d is joining an existing cluster
I20260928 14:43:12.447729 16108 master_runner.cc:432] Initiating AddMaster RPC to add 127.9.33.124:40843
I20260928 14:43:12.456192 16200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.448846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38209 (local address 127.0.0.1:46296)
0928 14:43:12.449193 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:12.449202 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:12.449280 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:12.449534 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:12.449535 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:12.449854 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:12.449942 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.449944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.450850 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.450851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.451791 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.451793 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.452607 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.452608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.452656 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.453094 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:12.453098 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:12.453417 (+ 319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:12.455052 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.455053 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.455054 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.455243 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.455448 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.455449 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.455449 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.455515 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.455831 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:12.455832 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:12.455880 (+ 48us) client_negotiation.cc:769] Sending connection context
0928 14:43:12.455968 (+ 88us) client_negotiation.cc:240] Negotiation successful
0928 14:43:12.456075 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"mutex_wait_us":35,"thread_start_us":110,"threads_started":1}
I20260928 14:43:12.456180 15498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.448872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46296 (local address 127.9.33.126:38209)
0928 14:43:12.448927 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:12.448930 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:12.449251 (+ 321us) server_negotiation.cc:407] Connection header received
0928 14:43:12.449401 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:12.449402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:12.449416 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:12.449462 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:12.450035 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.450768 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.451890 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.452188 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.452227 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.453533 (+ 1306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:12.453534 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:12.453535 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:12.453542 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:12.454948 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.455318 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.455319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.455319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.455380 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.455587 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.455588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.455589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.455701 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:12.455847 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:12.455994 (+ 147us) server_negotiation.cc:299] Negotiation successful
0928 14:43:12.456072 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
Sep 28 14:43:12 dist-test-slave-pv0w krb5kdc[15410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606591, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.126@KRBTEST.COM for oryx/127.9.33.124@KRBTEST.COM
I20260928 14:43:12.479525 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 15517
I20260928 14:43:12.485742 16013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.458274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:40843 (local address 127.0.0.1:51018)
0928 14:43:12.458332 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:12.458342 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:12.458380 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:12.458637 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:12.458638 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:12.458789 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:12.458835 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.458837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.459525 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.459526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.460158 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.460160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.460636 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.460637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.460706 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.461151 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:12.461157 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:12.462432 (+ 1275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:12.483216 (+ 20784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.483218 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.483219 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.483474 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.483757 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.483758 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.483759 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.483827 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.485091 (+ 1264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:12.485093 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:12.485151 (+ 58us) client_negotiation.cc:769] Sending connection context
0928 14:43:12.485206 (+ 55us) client_negotiation.cc:240] Negotiation successful
0928 14:43:12.485604 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260928 14:43:12.485759 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.65:39055
--local_ip_for_outbound_sockets=127.9.33.65
--tserver_master_addrs=127.9.33.126:38209
--webserver_port=36105
--webserver_interface=127.9.33.65
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.65.keytab
--principal=oryx/127.9.33.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:41001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.33.126:38209,127.9.33.125:36321
--tserver_master_addrs=127.9.33.126:38209,127.9.33.125:36321,127.9.33.124:40843 with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:43:12.486306 16105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.449053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47114 (local address 127.9.33.125:36321)
0928 14:43:12.449106 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:12.449108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:12.449118 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:43:12.449142 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:12.449142 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:12.449153 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:12.449230 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:12.449656 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.450437 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.451574 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.451916 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.451962 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.481916 (+ 29954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:12.481918 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:12.481918 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:12.481927 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:12.483955 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.484365 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.484367 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.484367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.484424 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.484821 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.484823 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.484823 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.484989 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:12.485056 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:12.486098 (+ 1042us) server_negotiation.cc:299] Negotiation successful
0928 14:43:12.486186 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 14:43:12.486969 16201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.458339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51018 (local address 127.9.33.124:40843)
0928 14:43:12.458510 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:12.458512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:12.458517 (+ 5us) server_negotiation.cc:407] Connection header received
0928 14:43:12.458545 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:12.458545 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:12.458553 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:12.458621 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:12.458919 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.459436 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.460275 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.460564 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.460637 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.462539 (+ 1902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:12.462540 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:12.462541 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:12.462548 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:12.483091 (+ 20543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.483572 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.483573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.483574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.483638 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.483905 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.483906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.483907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.484995 (+ 1088us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:12.485941 (+ 946us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:12.485993 (+ 52us) server_negotiation.cc:299] Negotiation successful
0928 14:43:12.486870 (+ 877us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":118,"threads_started":1}
I20260928 14:43:12.487175 16199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.448841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.125:36321 (local address 127.0.0.1:47114)
0928 14:43:12.449010 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:12.449022 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:12.449081 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:12.449250 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:12.449252 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:12.449471 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:12.449557 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.449560 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.450541 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.450542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.451471 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.451474 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.452057 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.452058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.452121 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.452598 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:12.452604 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:12.481589 (+ 28985us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:12.484027 (+ 2438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.484028 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.484029 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.484279 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.484494 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.484495 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.484495 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.484746 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.485388 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:12.485389 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:12.485434 (+ 45us) client_negotiation.cc:769] Sending connection context
0928 14:43:12.485542 (+ 108us) client_negotiation.cc:240] Negotiation successful
0928 14:43:12.487087 (+ 1545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":620,"thread_start_us":89,"threads_started":1}
I20260928 14:43:12.487481 15454 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.9.33.124:40843: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d"
member_type: NON_VOTER
last_known_addr {
host: "127.9.33.124"
port: 40843
}
attrs {
promote: true
}
}
dest_uuid: "4cfad45886d34600a02df0ae697f86b2"
cas_config_opid_index: 5
I20260928 14:43:12.487797 15454 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: NON_VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: true } } peers { permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: NON_VOTER last_known_addr { host: "127.9.33.124" port: 40843 } attrs { promote: true } }
I20260928 14:43:12.488235 16016 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER ba6dfd46bca84cc5a9551e578c07703d (127.9.33.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: NON_VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: true } } peers { permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: NON_VOTER last_known_addr { host: "127.9.33.124" port: 40843 } attrs { promote: true } } }
I20260928 14:43:12.488320 16198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.449089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:40843 (local address 127.0.0.1:51008)
0928 14:43:12.449305 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:12.449314 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:12.449371 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:12.449541 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:12.449541 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:12.449847 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:12.449942 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.449943 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.450847 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.450848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.451791 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.451793 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.452262 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.452263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.452337 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.481685 (+ 29348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:12.481691 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:12.484401 (+ 2710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:12.486320 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.486321 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.486322 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.486532 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.486747 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.486748 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.486748 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.486803 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.487968 (+ 1165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:12.487969 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:12.488006 (+ 37us) client_negotiation.cc:769] Sending connection context
0928 14:43:12.488074 (+ 68us) client_negotiation.cc:240] Negotiation successful
0928 14:43:12.488183 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":51,"thread_start_us":72,"threads_started":1}
I20260928 14:43:12.488576 16179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.449228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51008 (local address 127.9.33.124:40843)
0928 14:43:12.449262 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:12.449263 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:12.449353 (+ 90us) server_negotiation.cc:407] Connection header received
0928 14:43:12.449454 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:12.449454 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:12.449462 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:12.449490 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:12.450026 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.450746 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.451888 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.452157 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.452193 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.484498 (+ 32305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:12.484500 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:12.484500 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:12.484506 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:12.486243 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.486621 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.486623 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.486623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.486679 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.487712 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.487738 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.487738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.487883 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:12.487946 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:12.488420 (+ 474us) server_negotiation.cc:299] Negotiation successful
0928 14:43:12.488500 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 14:43:12.489254 16016 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.9.33.124:40843
I20260928 14:43:12.489351 16016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4cfad45886d34600a02df0ae697f86b2" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: NON_VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: true } } peers { permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: NON_VOTER last_known_addr { host: "127.9.33.124" port: 40843 } attrs { promote: true } } }
I20260928 14:43:12.489419 16016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [sys.catalog]: This master's current role is: LEADER
I20260928 14:43:12.489722 16108 master.cc:537] Master@127.9.33.124:40843 shutting down...
W20260928 14:43:12.490612 15439 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 -> Peer ba6dfd46bca84cc5a9551e578c07703d (127.9.33.124:40843): Couldn't send request to peer ba6dfd46bca84cc5a9551e578c07703d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260928 14:43:12.493112 16108 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260928 14:43:12.493235 16108 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260928 14:43:12.493304 16108 tablet_replica.cc:330] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d: stopping tablet replica
I20260928 14:43:12.494031 16013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.489093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.125:36321 (local address 127.0.0.1:47124)
0928 14:43:12.489347 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:12.489359 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:12.489396 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:12.490158 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:12.490159 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:12.490161 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:43:12.490209 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.490211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.491224 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.491225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.492707 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.492711 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.493703 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.493704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.493786 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.493819 (+ 33us) client_negotiation.cc:769] Sending connection context
0928 14:43:12.493872 (+ 53us) client_negotiation.cc:240] Negotiation successful
0928 14:43:12.493927 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234}
I20260928 14:43:12.494249 16105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.489601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47124 (local address 127.9.33.125:36321)
0928 14:43:12.489638 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:12.489640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:12.489647 (+ 7us) server_negotiation.cc:407] Connection header received
0928 14:43:12.489670 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:12.489670 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:12.489695 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:12.489764 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:43:12.490281 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.491096 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.492809 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.493493 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.493572 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.493663 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:12.494146 (+ 483us) server_negotiation.cc:299] Negotiation successful
0928 14:43:12.494185 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 14:43:12.494529 16064 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [term 1 LEARNER]: Refusing update from remote peer 4cfad45886d34600a02df0ae697f86b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260928 14:43:12.494849 16014 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: NON_VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260928 14:43:12.495517 16064 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER ba6dfd46bca84cc5a9551e578c07703d (127.9.33.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: NON_VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: true } } peers { permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: NON_VOTER last_known_addr { host: "127.9.33.124" port: 40843 } attrs { promote: true } } }
I20260928 14:43:12.496938 16093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4cfad45886d34600a02df0ae697f86b2. Latest consensus state: current_term: 1 leader_uuid: "4cfad45886d34600a02df0ae697f86b2" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: NON_VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: true } } peers { permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: NON_VOTER last_known_addr { host: "127.9.33.124" port: 40843 } attrs { promote: true } } }
I20260928 14:43:12.497052 16093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [sys.catalog]: This master's current role is: LEARNER
I20260928 14:43:12.497095 16093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4cfad45886d34600a02df0ae697f86b2" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: NON_VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: true } } peers { permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: NON_VOTER last_known_addr { host: "127.9.33.124" port: 40843 } attrs { promote: true } } }
I20260928 14:43:12.497169 16093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [sys.catalog]: This master's current role is: LEARNER
I20260928 14:43:12.497342 16108 master.cc:552] Master@127.9.33.124:40843 shutdown complete.
I20260928 14:43:12.497392 16108 master_runner.cc:305] Clearing existing system tablet
I20260928 14:43:12.497606 16108 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:12.497668 16108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:12.498533 16108 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:12.498934 16014 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2: attempting to promote NON_VOTER 39a978772cff40a6882ec08ae1ae2f52 to VOTER
I20260928 14:43:12.499239 16206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:12.499244 16014 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: false } } peers { permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: NON_VOTER last_known_addr { host: "127.9.33.124" port: 40843 } attrs { promote: true } }
I20260928 14:43:12.499475 16108 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:12.499526 16108 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260928 14:43:12.499549 16108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-2/wal
uuid: "ba6dfd46bca84cc5a9551e578c07703d"
format_stamp: "Formatted at 2026-09-28 14:43:12 on dist-test-slave-pv0w"
server_key: "be2fa3866172817b5830f05e5a258c53"
server_key_iv: "86e1bc4d1d56ebdcd2a7a1aa5e658053"
server_key_version: "encryptionkey@0"
I20260928 14:43:12.499903 16064 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [term 1 LEARNER]: Refusing update from remote peer 4cfad45886d34600a02df0ae697f86b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260928 14:43:12.500087 16013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.499952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:40843 (local address 127.0.0.1:51024)
0928 14:43:12.500002 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:12.500053 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.124:40843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260928 14:43:12.500253 16016 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260928 14:43:12.500696 16013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.500609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:40843 (local address 127.0.0.1:51028)
0928 14:43:12.500645 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:12.500668 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.124:40843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260928 14:43:12.500767 15439 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 -> Peer ba6dfd46bca84cc5a9551e578c07703d (127.9.33.124:40843): Couldn't send request to peer ba6dfd46bca84cc5a9551e578c07703d. Status: Network error: Client connection negotiation failed: client connection to 127.9.33.124:40843: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260928 14:43:12.501070 16016 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 39a978772cff40a6882ec08ae1ae2f52 (127.9.33.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: false } } peers { permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: NON_VOTER last_known_addr { host: "127.9.33.124" port: 40843 } attrs { promote: true } } }
I20260928 14:43:12.501236 16108 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:43:12.502028 16016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4cfad45886d34600a02df0ae697f86b2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: false } } peers { permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: NON_VOTER last_known_addr { host: "127.9.33.124" port: 40843 } attrs { promote: true } } }
I20260928 14:43:12.502100 16016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [sys.catalog]: This master's current role is: LEADER
I20260928 14:43:12.502146 16016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4cfad45886d34600a02df0ae697f86b2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: false } } peers { permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: NON_VOTER last_known_addr { host: "127.9.33.124" port: 40843 } attrs { promote: true } } }
I20260928 14:43:12.502182 16016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [sys.catalog]: This master's current role is: LEADER
I20260928 14:43:12.502599 16064 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 39a978772cff40a6882ec08ae1ae2f52 (127.9.33.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: false } } peers { permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: NON_VOTER last_known_addr { host: "127.9.33.124" port: 40843 } attrs { promote: true } } }
I20260928 14:43:12.502854 16108 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260928 14:43:12.502923 16108 log.cc:1196] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260928 14:43:12.503144 16108 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d: Deleting consensus metadata
I20260928 14:43:12.503253 16108 master_runner.cc:315] Copying system tablet from 127.9.33.126:38209
I20260928 14:43:12.503813 16204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4cfad45886d34600a02df0ae697f86b2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: false } } peers { permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: NON_VOTER last_known_addr { host: "127.9.33.124" port: 40843 } attrs { promote: true } } }
I20260928 14:43:12.503894 16108 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d: tablet copy: Beginning tablet copy session from remote peer at address 127.9.33.126:38209
I20260928 14:43:12.503990 16204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:43:12.511605 15498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.504305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46302 (local address 127.9.33.126:38209)
0928 14:43:12.504349 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:12.504351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:12.504679 (+ 328us) server_negotiation.cc:407] Connection header received
0928 14:43:12.504705 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:12.504706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:12.504718 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:12.504768 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:12.505162 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.505973 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.507063 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.507395 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.507469 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.508706 (+ 1237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:12.508708 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:12.508708 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:12.508716 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:12.510362 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.510769 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.510770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.510770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.510831 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.511045 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.511046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.511046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.511184 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:12.511250 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:12.511420 (+ 170us) server_negotiation.cc:299] Negotiation successful
0928 14:43:12.511508 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 14:43:12.511636 16212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.504383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38209 (local address 127.0.0.1:46302)
0928 14:43:12.504554 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:12.504566 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:12.504637 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:12.504786 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:12.504786 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:12.505013 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:12.505081 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.505083 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.506067 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.506069 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.506969 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.506973 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.507476 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.507477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.507549 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.508153 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:12.508159 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:12.508599 (+ 440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:12.510453 (+ 1854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.510455 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.510455 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.510686 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.510906 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.510907 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.510907 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.510975 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.511254 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:12.511255 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:12.511307 (+ 52us) client_negotiation.cc:769] Sending connection context
0928 14:43:12.511379 (+ 72us) client_negotiation.cc:240] Negotiation successful
0928 14:43:12.511497 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":102,"threads_started":1}
I20260928 14:43:12.512315 15475 tablet_copy_service.cc:140] P 4cfad45886d34600a02df0ae697f86b2: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer ba6dfd46bca84cc5a9551e578c07703d ({username='oryx', principal='oryx/127.9.33.124@KRBTEST.COM'} at 127.0.0.1:46302)
I20260928 14:43:12.512408 15475 tablet_copy_service.cc:161] P 4cfad45886d34600a02df0ae697f86b2: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer ba6dfd46bca84cc5a9551e578c07703d at {username='oryx', principal='oryx/127.9.33.124@KRBTEST.COM'} at 127.0.0.1:46302: session id = ba6dfd46bca84cc5a9551e578c07703d-00000000000000000000000000000000
I20260928 14:43:12.513033 15475 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2: Tablet Copy: opened 0 blocks and 1 log segments
I20260928 14:43:12.513573 16108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260928 14:43:12.514983 16108 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d: tablet copy: Starting download of 0 data blocks...
I20260928 14:43:12.515139 16108 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d: tablet copy: Starting download of 1 WAL segments...
I20260928 14:43:12.516196 16108 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260928 14:43:12.517122 15475 tablet_copy_service.cc:342] P 4cfad45886d34600a02df0ae697f86b2: Request end of tablet copy session ba6dfd46bca84cc5a9551e578c07703d-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.9.33.124@KRBTEST.COM'} at 127.0.0.1:46302
I20260928 14:43:12.517174 15475 tablet_copy_service.cc:433] P 4cfad45886d34600a02df0ae697f86b2: ending tablet copy session ba6dfd46bca84cc5a9551e578c07703d-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer ba6dfd46bca84cc5a9551e578c07703d
I20260928 14:43:12.520059 16108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:12.521597 16216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:12.521647 16213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:12.521737 16214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:12.522073 16108 server_base.cc:1034] running on GCE node
I20260928 14:43:12.522171 16108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:12.522313 16108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:12.523456 16108 hybrid_clock.cc:648] HybridClock initialized: now 1790606592523448 us; error 26 us; skew 500 ppm
Sep 28 14:43:12 dist-test-slave-pv0w krb5kdc[15410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606592, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:12.525035 16108 init.cc:377] Logged in from keytab as oryx/127.9.33.124@KRBTEST.COM (short username oryx)
I20260928 14:43:12.525307 16108 webserver.cc:460] Webserver started at http://127.9.33.124:41271/ using document root <none> and password file <none>
I20260928 14:43:12.525434 16108 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:12.525470 16108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:12.526315 16108 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:12.526927 16223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:12.527103 16108 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:43:12.527161 16108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-2/wal
uuid: "ba6dfd46bca84cc5a9551e578c07703d"
format_stamp: "Formatted at 2026-09-28 14:43:12 on dist-test-slave-pv0w"
server_key: "be2fa3866172817b5830f05e5a258c53"
server_key_iv: "86e1bc4d1d56ebdcd2a7a1aa5e658053"
server_key_version: "encryptionkey@0"
I20260928 14:43:12.527237 16108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:12.546169 16108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:12.548123 16108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:12.551362 16108 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.124:40843
I20260928 14:43:12.551820 16108 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/master-2/data/info.pb
I20260928 14:43:12.552876 16276 sys_catalog.cc:263] Verifying existing consensus state
I20260928 14:43:12.553417 16276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d: Bootstrap starting.
I20260928 14:43:12.556715 16276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 14:43:12.557026 16276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d: Bootstrap complete.
I20260928 14:43:12.557260 16276 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: false } } peers { permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: NON_VOTER last_known_addr { host: "127.9.33.124" port: 40843 } attrs { promote: true } }
I20260928 14:43:12.557359 16276 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ba6dfd46bca84cc5a9551e578c07703d, State: Initialized, Role: LEARNER
I20260928 14:43:12.557401 16276 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: false } } peers { permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: NON_VOTER last_known_addr { host: "127.9.33.124" port: 40843 } attrs { promote: true } }
I20260928 14:43:12.557601 16276 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:43:12.557619 16279 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: false } } peers { permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: NON_VOTER last_known_addr { host: "127.9.33.124" port: 40843 } attrs { promote: true } } }
I20260928 14:43:12.557832 16279 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d [sys.catalog]: This master's current role is: LEARNER
I20260928 14:43:12.559128 16290 catalog_manager.cc:1259] Loaded cluster ID: 063a364e0f4846ca91dc5cd374c69e34
I20260928 14:43:12.559218 16290 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d: loading cluster ID for follower catalog manager: success
I20260928 14:43:12.559391 16275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.124:40843 every 8 connection(s)
I20260928 14:43:12.560539 16290 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d: acquiring CA information for follower catalog manager: success
I20260928 14:43:12.560879 16290 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260928 14:43:12.589078 16202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:12.589319 16202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:12.589344 16202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:12.590543 16202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:12.590587 16202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:12.590615 16202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.65
I20260928 14:43:12.592195 16202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:41001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.65.keytab
--principal=oryx/127.9.33.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.65:39055
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.33.65
--webserver_port=36105
--tserver_master_addrs=127.9.33.126:38209,127.9.33.125:36321,127.9.33.124:40843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.65
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:12.592446 16202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:12.592715 16202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:12.597952 16298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:12.597935 16296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:12.598186 16202 server_base.cc:1034] running on GCE node
W20260928 14:43:12.597932 16295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:12.598423 16202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:12.598623 16202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:12.599778 16202 hybrid_clock.cc:648] HybridClock initialized: now 1790606592599761 us; error 30 us; skew 500 ppm
Sep 28 14:43:12 dist-test-slave-pv0w krb5kdc[15410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606592, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:12.602310 16202 init.cc:377] Logged in from keytab as oryx/127.9.33.65@KRBTEST.COM (short username oryx)
I20260928 14:43:12.602716 16202 webserver.cc:460] Webserver started at http://127.9.33.65:36105/ using document root <none> and password file <none>
I20260928 14:43:12.602900 16202 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:12.602954 16202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:12.604492 16202 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:43:12.605087 16305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:12.605268 16202 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:12.605342 16202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-0/wal
uuid: "dc15cdcdf4b145588845d6ce64504136"
format_stamp: "Formatted at 2026-09-28 14:43:11 on dist-test-slave-pv0w"
server_key: "6dde13c35c8dbbdd70fd8a26245fd8a7"
server_key_iv: "186301989e4c4a284793d2c6f12533ff"
server_key_version: "encryptionkey@0"
I20260928 14:43:12.605759 16202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:12.622807 16202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:12.625272 16202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:12.625437 16202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:12.625713 16202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:43:12.626070 16202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:43:12.626128 16202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:12.626165 16202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:43:12.626191 16202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:12.632174 16202 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.65:39055
I20260928 14:43:12.632215 16418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.65:39055 every 8 connection(s)
I20260928 14:43:12.632599 16202 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
I20260928 14:43:12.633704 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 16202
I20260928 14:43:12.633893 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 15651
I20260928 14:43:12.640764 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.66:37471
--local_ip_for_outbound_sockets=127.9.33.66
--tserver_master_addrs=127.9.33.126:38209
--webserver_port=35435
--webserver_interface=127.9.33.66
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.66.keytab
--principal=oryx/127.9.33.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:41001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.33.126:38209,127.9.33.125:36321
--tserver_master_addrs=127.9.33.126:38209,127.9.33.125:36321,127.9.33.124:40843 with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 14:43:12 dist-test-slave-pv0w krb5kdc[15410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606592, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM
Sep 28 14:43:12 dist-test-slave-pv0w krb5kdc[15410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606592, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.125@KRBTEST.COM
Sep 28 14:43:12 dist-test-slave-pv0w krb5kdc[15410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606592, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.124@KRBTEST.COM
I20260928 14:43:12.646461 15498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.633916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:46615 (local address 127.9.33.126:38209)
0928 14:43:12.633981 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:12.633983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:12.636625 (+ 2642us) server_negotiation.cc:407] Connection header received
0928 14:43:12.636664 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:12.636665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:12.636677 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:12.636756 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:12.637055 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.637685 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.638887 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.639191 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.639234 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.642747 (+ 3513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:12.642749 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:12.642749 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:12.642759 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:12.644607 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.645159 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.645160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.645161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.645224 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.645555 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.645556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.645556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.645719 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:12.645769 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:12.646201 (+ 432us) server_negotiation.cc:299] Negotiation successful
0928 14:43:12.646301 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260928 14:43:12.646714 16423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.633403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:40047 (local address 127.9.33.124:40843)
0928 14:43:12.633550 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:12.633553 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:12.635167 (+ 1614us) server_negotiation.cc:407] Connection header received
0928 14:43:12.635406 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:12.635407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:12.635420 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:12.635463 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:12.637435 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.638538 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.639879 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.640166 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.640238 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.643496 (+ 3258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:12.643498 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:12.643498 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:12.643507 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:12.645342 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.645779 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.645780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.645781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.645848 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.646058 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.646059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.646059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.646209 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:12.646250 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:12.646483 (+ 233us) server_negotiation.cc:299] Negotiation successful
0928 14:43:12.646584 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":60,"threads_started":1}
I20260928 14:43:12.646885 16105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.633717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:59707 (local address 127.9.33.125:36321)
0928 14:43:12.633766 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:12.633768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:12.634661 (+ 893us) server_negotiation.cc:407] Connection header received
0928 14:43:12.635263 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:12.635264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:12.635276 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:12.635313 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:12.636490 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.637314 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.638883 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.639191 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.639234 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.643307 (+ 4073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:12.643310 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:12.643310 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:12.643319 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:12.645682 (+ 2363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.646067 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.646068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.646069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.646123 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.646320 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.646321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.646321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.646468 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:12.646510 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:12.646717 (+ 207us) server_negotiation.cc:299] Negotiation successful
0928 14:43:12.646799 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260928 14:43:12.647323 16426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.633412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:40843 (local address 127.9.33.65:40047)
0928 14:43:12.633821 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:12.635107 (+ 1286us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:12.635192 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:12.635540 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:12.635541 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:12.636036 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:12.636378 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.636384 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.638636 (+ 2252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.638637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.639747 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.639750 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.640294 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.640295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.640375 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.640739 (+ 364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:12.640744 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:12.643369 (+ 2625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:12.645438 (+ 2069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.645439 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.645440 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.645672 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.645925 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.645926 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.645926 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.645981 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.646296 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:12.646297 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:12.646342 (+ 45us) client_negotiation.cc:769] Sending connection context
0928 14:43:12.646850 (+ 508us) client_negotiation.cc:240] Negotiation successful
0928 14:43:12.647062 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"mutex_wait_us":40,"thread_start_us":122,"threads_started":1}
I20260928 14:43:12.647323 16425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.633714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38209 (local address 127.9.33.65:46615)
0928 14:43:12.636526 (+ 2812us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:12.636537 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:12.636615 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:12.636759 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:12.636760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:12.636917 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:12.636971 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.636973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.637776 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.637777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.638778 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.638781 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.639330 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.639331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.639500 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.640214 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:12.640221 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:12.642617 (+ 2396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:12.644702 (+ 2085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.644711 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.644714 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.645059 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.645302 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.645303 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.645303 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.645469 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.645820 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:12.645827 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:12.646026 (+ 199us) client_negotiation.cc:769] Sending connection context
0928 14:43:12.647034 (+ 1008us) client_negotiation.cc:240] Negotiation successful
0928 14:43:12.647108 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2787,"mutex_wait_us":9,"thread_start_us":63,"threads_started":1}
I20260928 14:43:12.647846 16424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.633413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.125:36321 (local address 127.9.33.65:59707)
0928 14:43:12.633861 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:12.634397 (+ 536us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:12.635174 (+ 777us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:12.635420 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:12.635425 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:12.636043 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:12.636378 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.636384 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.637438 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.637439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.638778 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.638781 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.639330 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.639331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.639500 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.640164 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:12.640176 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:12.642959 (+ 2783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:12.645768 (+ 2809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.645769 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.645769 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.645984 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.646194 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.646195 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.646196 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.646248 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.646557 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:12.646557 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:12.646610 (+ 53us) client_negotiation.cc:769] Sending connection context
0928 14:43:12.646971 (+ 361us) client_negotiation.cc:240] Negotiation successful
0928 14:43:12.647392 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":365,"thread_start_us":95,"threads_started":1}
I20260928 14:43:12.647926 16420 heartbeater.cc:347] Connected to a master server at 127.9.33.124:40843
I20260928 14:43:12.648027 16420 heartbeater.cc:464] Registering TS with master...
I20260928 14:43:12.648290 16419 heartbeater.cc:347] Connected to a master server at 127.9.33.126:38209
I20260928 14:43:12.648339 16419 heartbeater.cc:464] Registering TS with master...
I20260928 14:43:12.648413 16420 heartbeater.cc:511] Master 127.9.33.124:40843 requested a full tablet report, sending...
I20260928 14:43:12.648413 16419 heartbeater.cc:511] Master 127.9.33.126:38209 requested a full tablet report, sending...
I20260928 14:43:12.648550 16421 heartbeater.cc:347] Connected to a master server at 127.9.33.125:36321
I20260928 14:43:12.648597 16421 heartbeater.cc:464] Registering TS with master...
I20260928 14:43:12.648674 16421 heartbeater.cc:511] Master 127.9.33.125:36321 requested a full tablet report, sending...
I20260928 14:43:12.648898 15454 ts_manager.cc:194] Re-registered known tserver with Master: dc15cdcdf4b145588845d6ce64504136 (127.9.33.65:39055)
I20260928 14:43:12.648993 16240 ts_manager.cc:194] Registered new tserver with Master: dc15cdcdf4b145588845d6ce64504136 (127.9.33.65:39055)
I20260928 14:43:12.649394 15454 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.33.65@KRBTEST.COM'} at 127.9.33.65:46615
I20260928 14:43:12.649812 16054 ts_manager.cc:194] Registered new tserver with Master: dc15cdcdf4b145588845d6ce64504136 (127.9.33.65:39055)
W20260928 14:43:12.727608 16427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:12.727916 16427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:12.727962 16427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:12.729249 16427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:12.729315 16427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:12.729360 16427 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.66
I20260928 14:43:12.730907 16427 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:41001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.66.keytab
--principal=oryx/127.9.33.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.66:37471
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.33.66
--webserver_port=35435
--tserver_master_addrs=127.9.33.126:38209,127.9.33.125:36321,127.9.33.124:40843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.66
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:12.731210 16427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:12.731463 16427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:12.737206 16435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:12.737200 16432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:12.737207 16433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:12.737696 16427 server_base.cc:1034] running on GCE node
I20260928 14:43:12.737951 16427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:12.738234 16427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:12.739441 16427 hybrid_clock.cc:648] HybridClock initialized: now 1790606592739415 us; error 36 us; skew 500 ppm
Sep 28 14:43:12 dist-test-slave-pv0w krb5kdc[15410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606592, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:12.742187 16427 init.cc:377] Logged in from keytab as oryx/127.9.33.66@KRBTEST.COM (short username oryx)
I20260928 14:43:12.742699 16427 webserver.cc:460] Webserver started at http://127.9.33.66:35435/ using document root <none> and password file <none>
I20260928 14:43:12.742945 16427 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:12.743039 16427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:12.744585 16427 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:12.745247 16442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:12.745431 16427 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:43:12.745539 16427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-1/wal
uuid: "b5fdff97460e49f5acef24e0105aab26"
format_stamp: "Formatted at 2026-09-28 14:43:11 on dist-test-slave-pv0w"
server_key: "a420c9ef74f63fee892ea11925ae97ae"
server_key_iv: "f015b9587c00e6d1c6d93091a35946f7"
server_key_version: "encryptionkey@0"
I20260928 14:43:12.745954 16427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:12.757525 16427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:12.759858 16427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:12.760036 16427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:12.760293 16427 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:43:12.760648 16427 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:43:12.760707 16427 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:12.760753 16427 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:43:12.760812 16427 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:12.766969 16427 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.66:37471
I20260928 14:43:12.767022 16555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.66:37471 every 8 connection(s)
I20260928 14:43:12.767434 16427 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
Sep 28 14:43:12 dist-test-slave-pv0w krb5kdc[15410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606592, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM
Sep 28 14:43:12 dist-test-slave-pv0w krb5kdc[15410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606592, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.125@KRBTEST.COM
Sep 28 14:43:12 dist-test-slave-pv0w krb5kdc[15410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606592, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.124@KRBTEST.COM
I20260928 14:43:12.776827 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 16427
I20260928 14:43:12.777021 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 15785
I20260928 14:43:12.781446 15498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.768738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:41583 (local address 127.9.33.126:38209)
0928 14:43:12.768813 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:12.768815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:12.768959 (+ 144us) server_negotiation.cc:407] Connection header received
0928 14:43:12.769334 (+ 375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:12.769335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:12.769348 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:12.769394 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:12.770210 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.770990 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.772172 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.772467 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.772524 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.774785 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:12.774786 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:12.774787 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:12.774793 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:12.776438 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.776896 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.776898 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.776898 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.776964 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.780536 (+ 3572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.780538 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.780538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.780711 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:12.780776 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:12.781167 (+ 391us) server_negotiation.cc:299] Negotiation successful
0928 14:43:12.781327 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260928 14:43:12.781587 16423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.768560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:57599 (local address 127.9.33.124:40843)
0928 14:43:12.768637 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:12.768640 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:12.768651 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:43:12.769334 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:12.769335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:12.769351 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:12.769382 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:12.770212 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.770980 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.772567 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.772855 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.772921 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.775871 (+ 2950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:12.775872 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:12.775873 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:12.775881 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:12.777482 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.780678 (+ 3196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.780679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.780680 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.780741 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.780963 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.780964 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.780964 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.781108 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:12.781160 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:12.781346 (+ 186us) server_negotiation.cc:299] Negotiation successful
0928 14:43:12.781446 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260928 14:43:12.781875 16105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.768269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:51601 (local address 127.9.33.125:36321)
0928 14:43:12.768628 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:12.768631 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:12.768828 (+ 197us) server_negotiation.cc:407] Connection header received
0928 14:43:12.769315 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:12.769316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:12.769330 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:12.769373 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:12.770193 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.770701 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.771743 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.771963 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.772048 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.775549 (+ 3501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:12.775550 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:12.775550 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:12.775556 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:12.778487 (+ 2931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.780975 (+ 2488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.780976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.780976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.781038 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.781243 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.781244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.781244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.781387 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:12.781431 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:12.781675 (+ 244us) server_negotiation.cc:299] Negotiation successful
0928 14:43:12.781790 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328}
I20260928 14:43:12.782248 16561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.768289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:40843 (local address 127.9.33.66:57599)
0928 14:43:12.768467 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:12.768505 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:12.769205 (+ 700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:12.769458 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:12.769460 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:12.769829 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:12.770099 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.770106 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.771082 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.771083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.772045 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.772047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.772975 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.772976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.773039 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.773436 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:12.773441 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:12.775760 (+ 2319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:12.777564 (+ 1804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.777565 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.777565 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.780592 (+ 3027us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.780818 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.780819 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.780820 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.780883 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.781194 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:12.781195 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:12.781240 (+ 45us) client_negotiation.cc:769] Sending connection context
0928 14:43:12.781712 (+ 472us) client_negotiation.cc:240] Negotiation successful
0928 14:43:12.781911 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":59,"thread_start_us":100,"threads_started":1}
I20260928 14:43:12.782253 16562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.768289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.125:36321 (local address 127.9.33.66:51601)
0928 14:43:12.768772 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:12.768781 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:12.769210 (+ 429us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:12.769452 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:12.769457 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:12.769850 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:12.770098 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.770103 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.770808 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.770808 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.771623 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.771625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.772061 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.772062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.772212 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.772826 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:12.772837 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:12.775186 (+ 2349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:12.778591 (+ 3405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.778592 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.778592 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.780884 (+ 2292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.781115 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.781116 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.781116 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.781174 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.781482 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:12.781483 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:12.781537 (+ 54us) client_negotiation.cc:769] Sending connection context
0928 14:43:12.781947 (+ 410us) client_negotiation.cc:240] Negotiation successful
0928 14:43:12.782075 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"mutex_wait_us":12,"spinlock_wait_cycles":8576,"thread_start_us":99,"threads_started":1}
I20260928 14:43:12.782846 16558 heartbeater.cc:347] Connected to a master server at 127.9.33.125:36321
I20260928 14:43:12.782954 16560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.768601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38209 (local address 127.9.33.66:41583)
0928 14:43:12.768887 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:12.768894 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:12.769206 (+ 312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:12.769466 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:12.769466 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:12.769932 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:12.770099 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.770106 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.771082 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.771083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.772045 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.772047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.772549 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.772550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.772614 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.772964 (+ 350us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:12.772976 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:12.774681 (+ 1705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:12.776501 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.776508 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.776511 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.776821 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.777044 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.777045 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.777046 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.780448 (+ 3402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.780807 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:12.780816 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:12.781003 (+ 187us) client_negotiation.cc:769] Sending connection context
0928 14:43:12.782267 (+ 1264us) client_negotiation.cc:240] Negotiation successful
0928 14:43:12.782783 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":56,"thread_start_us":57,"threads_started":1}
I20260928 14:43:12.782969 16558 heartbeater.cc:464] Registering TS with master...
I20260928 14:43:12.783151 16558 heartbeater.cc:511] Master 127.9.33.125:36321 requested a full tablet report, sending...
I20260928 14:43:12.783599 16054 ts_manager.cc:194] Registered new tserver with Master: b5fdff97460e49f5acef24e0105aab26 (127.9.33.66:37471)
I20260928 14:43:12.783625 16557 heartbeater.cc:347] Connected to a master server at 127.9.33.124:40843
I20260928 14:43:12.783668 16557 heartbeater.cc:464] Registering TS with master...
I20260928 14:43:12.783756 16557 heartbeater.cc:511] Master 127.9.33.124:40843 requested a full tablet report, sending...
I20260928 14:43:12.784080 16240 ts_manager.cc:194] Registered new tserver with Master: b5fdff97460e49f5acef24e0105aab26 (127.9.33.66:37471)
I20260928 14:43:12.784232 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.67:34405
--local_ip_for_outbound_sockets=127.9.33.67
--tserver_master_addrs=127.9.33.126:38209
--webserver_port=38031
--webserver_interface=127.9.33.67
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.67.keytab
--principal=oryx/127.9.33.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:41001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.33.126:38209,127.9.33.125:36321
--tserver_master_addrs=127.9.33.126:38209,127.9.33.125:36321,127.9.33.124:40843 with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:43:12.785329 16556 heartbeater.cc:347] Connected to a master server at 127.9.33.126:38209
I20260928 14:43:12.785400 16556 heartbeater.cc:464] Registering TS with master...
I20260928 14:43:12.785481 16556 heartbeater.cc:511] Master 127.9.33.126:38209 requested a full tablet report, sending...
I20260928 14:43:12.785769 15454 ts_manager.cc:194] Re-registered known tserver with Master: b5fdff97460e49f5acef24e0105aab26 (127.9.33.66:37471)
I20260928 14:43:12.786352 15454 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.33.66@KRBTEST.COM'} at 127.9.33.66:41583
W20260928 14:43:12.870217 16563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:12.870543 16563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:12.870589 16563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:12.871887 16563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:12.871966 16563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:12.872017 16563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.67
I20260928 14:43:12.873667 16563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:41001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.67.keytab
--principal=oryx/127.9.33.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.67:34405
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.33.67
--webserver_port=38031
--tserver_master_addrs=127.9.33.126:38209,127.9.33.125:36321,127.9.33.124:40843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.67
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:12.873936 16563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:12.874195 16563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:12.879767 16569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:12.879770 16571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:12.879829 16568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:12.879932 16563 server_base.cc:1034] running on GCE node
I20260928 14:43:12.880252 16563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:12.880479 16563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:12.881623 16563 hybrid_clock.cc:648] HybridClock initialized: now 1790606592881611 us; error 30 us; skew 500 ppm
Sep 28 14:43:12 dist-test-slave-pv0w krb5kdc[15410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606592, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:12.884343 16563 init.cc:377] Logged in from keytab as oryx/127.9.33.67@KRBTEST.COM (short username oryx)
I20260928 14:43:12.884840 16563 webserver.cc:460] Webserver started at http://127.9.33.67:38031/ using document root <none> and password file <none>
I20260928 14:43:12.885075 16563 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:12.885134 16563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:12.886555 16563 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:43:12.887267 16578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:12.887501 16563 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260928 14:43:12.887612 16563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-2/wal
uuid: "71899b6c516945508de4a4770b126e5f"
format_stamp: "Formatted at 2026-09-28 14:43:11 on dist-test-slave-pv0w"
server_key: "c8a7f79f3df858481c9f3df30099e540"
server_key_iv: "f84f062f41d706b68171bf5e06957ffd"
server_key_version: "encryptionkey@0"
I20260928 14:43:12.888091 16563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:12.898463 16563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:12.900719 16563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:12.900923 16563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:12.901177 16563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:43:12.901527 16563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:43:12.901584 16563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:12.901628 16563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:43:12.901656 16563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:12.907833 16563 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.67:34405
I20260928 14:43:12.907917 16691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.67:34405 every 8 connection(s)
I20260928 14:43:12.908246 16563 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
I20260928 14:43:12.909464 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 16563
Sep 28 14:43:12 dist-test-slave-pv0w krb5kdc[15410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606592, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.125@KRBTEST.COM
Sep 28 14:43:12 dist-test-slave-pv0w krb5kdc[15410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606592, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.124@KRBTEST.COM
I20260928 14:43:12.920970 16704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.910785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46316 (local address 127.9.33.126:38209)
0928 14:43:12.910936 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:12.910938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:12.911114 (+ 176us) server_negotiation.cc:407] Connection header received
0928 14:43:12.911209 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:12.911210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:12.911221 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:12.911269 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:12.912187 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.913000 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.914070 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.914366 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.914405 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.916410 (+ 2005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:12.916411 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:12.916412 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:12.916419 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:12.918530 (+ 2111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.918892 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.918893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.918894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.918954 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.919174 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.919175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.919175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.919765 (+ 590us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:12.919849 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:12.920320 (+ 471us) server_negotiation.cc:299] Negotiation successful
0928 14:43:12.920411 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":108,"threads_started":1}
I20260928 14:43:12.922530 16105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.909907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:40667 (local address 127.9.33.125:36321)
0928 14:43:12.909956 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:12.909958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:12.909967 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:43:12.910989 (+ 1022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:12.910990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:12.911002 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:12.911051 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:12.912280 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.913003 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.914500 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.914833 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.914897 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.919065 (+ 4168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:12.919067 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:12.919067 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:12.919074 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:12.920851 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.921373 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.921375 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.921375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.921435 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.921709 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.921710 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.921710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.921837 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:12.921896 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:12.922342 (+ 446us) server_negotiation.cc:299] Negotiation successful
0928 14:43:12.922428 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260928 14:43:12.923789 16697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.909390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:40843 (local address 127.9.33.67:49253)
0928 14:43:12.909556 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:12.909567 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:12.910757 (+ 1190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:12.911212 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:12.911213 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:12.911842 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:12.912189 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.912195 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.913257 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.913258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.914416 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.914419 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.915968 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.915969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.916074 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.916551 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:12.916558 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:12.919530 (+ 2972us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:12.922223 (+ 2693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.922224 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.922225 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.922477 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.922881 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.922882 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.922882 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.922949 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.923267 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:12.923268 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:12.923317 (+ 49us) client_negotiation.cc:769] Sending connection context
0928 14:43:12.923400 (+ 83us) client_negotiation.cc:240] Negotiation successful
0928 14:43:12.923509 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":35,"thread_start_us":74,"threads_started":1}
I20260928 14:43:12.924127 16423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.909914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:49253 (local address 127.9.33.124:40843)
0928 14:43:12.910019 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:12.910021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:12.910030 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:43:12.910953 (+ 923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:12.910955 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:12.910970 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:12.911035 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:12.912304 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.913167 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.914512 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.915552 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.915903 (+ 351us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.920295 (+ 4392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:12.920296 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:12.920296 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:12.920302 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:12.922003 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.922567 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.922568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.922568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.922636 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.923026 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.923027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.923028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.923172 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:12.923275 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:12.923936 (+ 661us) server_negotiation.cc:299] Negotiation successful
0928 14:43:12.924032 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260928 14:43:12.924420 16698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.909626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.125:36321 (local address 127.9.33.67:40667)
0928 14:43:12.909738 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:12.909750 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:12.910901 (+ 1151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:12.911187 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:12.911191 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:12.911984 (+ 793us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:12.912189 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.912195 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.913124 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.913125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.914414 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.914416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.914968 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.914969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.915102 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.916356 (+ 1254us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:12.916368 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:12.918970 (+ 2602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:12.920936 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.920943 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.920956 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.921290 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.921504 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.921504 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.921505 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.921622 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.921923 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:12.921929 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:12.922090 (+ 161us) client_negotiation.cc:769] Sending connection context
0928 14:43:12.922194 (+ 104us) client_negotiation.cc:240] Negotiation successful
0928 14:43:12.923377 (+ 1183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":7,"thread_start_us":74,"threads_started":1}
I20260928 14:43:12.924567 16706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.911096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51034 (local address 127.9.33.124:40843)
0928 14:43:12.911201 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:12.911203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:12.911680 (+ 477us) server_negotiation.cc:407] Connection header received
0928 14:43:12.911708 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:12.911709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:12.911750 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:12.911813 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:12.912955 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.913749 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.914918 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.915204 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.915271 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.917842 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:12.917844 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:12.917845 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:12.917852 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:12.920214 (+ 2362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.922756 (+ 2542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.922757 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.922757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.922820 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.923786 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.923788 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.923788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.923963 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:12.924072 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:12.924329 (+ 257us) server_negotiation.cc:299] Negotiation successful
0928 14:43:12.924427 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":52,"threads_started":1}
I20260928 14:43:12.924679 16693 heartbeater.cc:347] Connected to a master server at 127.9.33.124:40843
I20260928 14:43:12.924778 16693 heartbeater.cc:464] Registering TS with master...
I20260928 14:43:12.924993 16693 heartbeater.cc:511] Master 127.9.33.124:40843 requested a full tablet report, sending...
I20260928 14:43:12.925541 16240 ts_manager.cc:194] Registered new tserver with Master: 71899b6c516945508de4a4770b126e5f (127.9.33.67:34405)
Sep 28 14:43:12 dist-test-slave-pv0w krb5kdc[15410](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606592, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM
I20260928 14:43:12.925364 15454 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46316:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:43:12.927627 15454 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:43:12.928517 16694 heartbeater.cc:347] Connected to a master server at 127.9.33.125:36321
I20260928 14:43:12.928581 16694 heartbeater.cc:464] Registering TS with master...
I20260928 14:43:12.928665 16694 heartbeater.cc:511] Master 127.9.33.125:36321 requested a full tablet report, sending...
I20260928 14:43:12.928850 15498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.909053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:53901 (local address 127.9.33.126:38209)
0928 14:43:12.909160 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:12.909163 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:12.909406 (+ 243us) server_negotiation.cc:407] Connection header received
0928 14:43:12.911345 (+ 1939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:12.911346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:12.911356 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:12.911384 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:12.915371 (+ 3987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.916210 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.922475 (+ 6265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.922735 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.922769 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.926077 (+ 3308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:12.926078 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:12.926078 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:12.926086 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:12.927628 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.928052 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.928053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.928054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.928116 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.928319 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.928320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.928321 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.928479 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:12.928539 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:12.928702 (+ 163us) server_negotiation.cc:299] Negotiation successful
0928 14:43:12.928778 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 14:43:12.929168 16696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.909010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38209 (local address 127.9.33.67:53901)
0928 14:43:12.909209 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:12.909246 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:12.911264 (+ 2018us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:12.911438 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:12.911439 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:12.911849 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:12.912188 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.912194 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.920028 (+ 7834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.920030 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.922382 (+ 2352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.922384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.922828 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.922828 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.922876 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.924531 (+ 1655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:12.924537 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:12.925985 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:12.927707 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.927709 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.927709 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.927937 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.928182 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:12.928183 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:12.928183 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:12.928238 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:12.928528 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:12.928529 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:12.928584 (+ 55us) client_negotiation.cc:769] Sending connection context
0928 14:43:12.928940 (+ 356us) client_negotiation.cc:240] Negotiation successful
0928 14:43:12.929060 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":29,"thread_start_us":116,"threads_started":1}
I20260928 14:43:12.929302 16054 ts_manager.cc:194] Registered new tserver with Master: 71899b6c516945508de4a4770b126e5f (127.9.33.67:34405)
I20260928 14:43:12.929453 16692 heartbeater.cc:347] Connected to a master server at 127.9.33.126:38209
I20260928 14:43:12.929494 16692 heartbeater.cc:464] Registering TS with master...
I20260928 14:43:12.929553 16692 heartbeater.cc:511] Master 127.9.33.126:38209 requested a full tablet report, sending...
I20260928 14:43:12.929817 15455 ts_manager.cc:194] Re-registered known tserver with Master: 71899b6c516945508de4a4770b126e5f (127.9.33.67:34405)
I20260928 14:43:12.930197 15455 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.33.67@KRBTEST.COM'} at 127.9.33.67:53901
I20260928 14:43:12.931661 16013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.918748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:40843 (local address 127.0.0.1:51044)
0928 14:43:12.918851 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:12.918862 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:12.918899 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:12.920821 (+ 1922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:12.920822 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:12.920824 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:43:12.920873 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.920874 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.928706 (+ 7832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.928707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.930033 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.930036 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.931335 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.931336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.931408 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.931412 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:43:12.931520 (+ 108us) client_negotiation.cc:240] Negotiation successful
0928 14:43:12.931578 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260928 14:43:12.931876 16709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.918979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51044 (local address 127.9.33.124:40843)
0928 14:43:12.919280 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:12.919282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:12.919290 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:43:12.919314 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:12.919315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:12.919371 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:12.921392 (+ 2021us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:43:12.921437 (+ 45us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.928489 (+ 7052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.930511 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.931238 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.931336 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.931429 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:12.931532 (+ 103us) server_negotiation.cc:299] Negotiation successful
0928 14:43:12.931615 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":67,"threads_started":1}
I20260928 14:43:12.932183 16250 raft_consensus.cc:1203] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260928 14:43:12.932657 16014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4cfad45886d34600a02df0ae697f86b2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: false } } peers { permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: NON_VOTER last_known_addr { host: "127.9.33.124" port: 40843 } attrs { promote: true } } }
I20260928 14:43:12.932768 16014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [sys.catalog]: This master's current role is: LEADER
I20260928 14:43:12.932901 16279 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4cfad45886d34600a02df0ae697f86b2. Latest consensus state: current_term: 1 leader_uuid: "4cfad45886d34600a02df0ae697f86b2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: false } } peers { permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: NON_VOTER last_known_addr { host: "127.9.33.124" port: 40843 } attrs { promote: true } } }
I20260928 14:43:12.933012 16279 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d [sys.catalog]: This master's current role is: LEARNER
I20260928 14:43:12.933990 16707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.911101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47130 (local address 127.9.33.125:36321)
0928 14:43:12.911488 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:12.911490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:12.915027 (+ 3537us) server_negotiation.cc:407] Connection header received
0928 14:43:12.915138 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:12.915139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:12.915152 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:12.915195 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:12.917962 (+ 2767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.918774 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.920044 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.920416 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.920466 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.930479 (+ 10013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:12.930481 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:12.930482 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:12.930490 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:12.932505 (+ 2015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.933011 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.933012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.933013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.933082 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:12.933314 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:12.933314 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:12.933315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:12.933501 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:12.933635 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:12.933783 (+ 148us) server_negotiation.cc:299] Negotiation successful
0928 14:43:12.933880 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":49,"threads_started":1}
I20260928 14:43:12.934506 16015 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2: attempting to promote NON_VOTER ba6dfd46bca84cc5a9551e578c07703d to VOTER
I20260928 14:43:12.934651 16015 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: false } } peers { permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 40843 } attrs { promote: false } }
I20260928 14:43:12.935161 16250 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d [term 1 LEARNER]: Refusing update from remote peer 4cfad45886d34600a02df0ae697f86b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260928 14:43:12.935153 16064 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [term 1 FOLLOWER]: Refusing update from remote peer 4cfad45886d34600a02df0ae697f86b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260928 14:43:12.935353 16014 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260928 14:43:12.935595 16014 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 40843 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260928 14:43:12.936142 16014 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, ba6dfd46bca84cc5a9551e578c07703d (127.9.33.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: false } } peers { permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 40843 } attrs { promote: false } } }
I20260928 14:43:12.937013 16014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "4cfad45886d34600a02df0ae697f86b2" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: false } } peers { permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 40843 } attrs { promote: false } } }
I20260928 14:43:12.937096 16014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [sys.catalog]: This master's current role is: LEADER
I20260928 14:43:12.937144 16014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4cfad45886d34600a02df0ae697f86b2" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: false } } peers { permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 40843 } attrs { promote: false } } }
I20260928 14:43:12.937188 16014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [sys.catalog]: This master's current role is: LEADER
I20260928 14:43:12.937229 16014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4cfad45886d34600a02df0ae697f86b2" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: false } } peers { permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 40843 } attrs { promote: false } } }
I20260928 14:43:12.937271 16014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2 [sys.catalog]: This master's current role is: LEADER
I20260928 14:43:12.937400 16250 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, ba6dfd46bca84cc5a9551e578c07703d (127.9.33.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: false } } peers { permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 40843 } attrs { promote: false } } }
I20260928 14:43:12.938519 16064 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, ba6dfd46bca84cc5a9551e578c07703d (127.9.33.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: false } } peers { permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 40843 } attrs { promote: false } } }
I20260928 14:43:12.938577 16279 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4cfad45886d34600a02df0ae697f86b2" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: false } } peers { permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 40843 } attrs { promote: false } } }
I20260928 14:43:12.938645 16279 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:43:12.939488 16204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4cfad45886d34600a02df0ae697f86b2" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: false } } peers { permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 40843 } attrs { promote: false } } }
I20260928 14:43:12.939584 16204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:43:12.942178 16013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.937727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:39055 (local address 127.0.0.1:36374)
0928 14:43:12.937942 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:12.937954 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:12.938008 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:12.938506 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:12.938508 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:12.938510 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:43:12.938567 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.938568 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.939947 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.939948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.941151 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.941154 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.941920 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.941921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.941992 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.941995 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:43:12.942053 (+ 58us) client_negotiation.cc:240] Negotiation successful
0928 14:43:12.942097 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"spinlock_wait_cycles":768,"thread_start_us":97,"threads_started":1}
I20260928 14:43:12.942193 16719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.938147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36374 (local address 127.9.33.65:39055)
0928 14:43:12.938296 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:12.938301 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:12.938312 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:43:12.938359 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:12.938378 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:12.938397 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:12.938644 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:43:12.938707 (+ 63us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.939844 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.941249 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.941826 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.941923 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.942013 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:12.942073 (+ 60us) server_negotiation.cc:299] Negotiation successful
0928 14:43:12.942110 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":84,"threads_started":1}
I20260928 14:43:12.943252 16353 tablet_service.cc:1467] Processing CreateTablet for tablet 96e6b7dd4eda4bd9ab0229a111233f58 (DEFAULT_TABLE table=test-table [id=a2a37e497696402093f9df5884633f0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:43:12.943616 16721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.937643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:37471 (local address 127.0.0.1:40204)
0928 14:43:12.939887 (+ 2244us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:12.939899 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:12.939966 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:12.940155 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:12.940155 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:12.940157 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:43:12.940208 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.940209 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.941366 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.941367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.942425 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.942428 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.943287 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.943288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.943363 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.943368 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:43:12.943426 (+ 58us) client_negotiation.cc:240] Negotiation successful
0928 14:43:12.943538 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2226,"mutex_wait_us":736,"thread_start_us":69,"threads_started":1}
I20260928 14:43:12.943660 16718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.937767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40204 (local address 127.9.33.66:37471)
0928 14:43:12.939032 (+ 1265us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:12.939037 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:12.939969 (+ 932us) server_negotiation.cc:407] Connection header received
0928 14:43:12.940043 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:12.940046 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:12.940063 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:12.940142 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:43:12.940264 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.941279 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.942514 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.943179 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.943283 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.943359 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:12.943440 (+ 81us) server_negotiation.cc:299] Negotiation successful
0928 14:43:12.943479 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1223,"thread_start_us":85,"threads_started":1}
I20260928 14:43:12.943907 16720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.938173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57490 (local address 127.9.33.67:34405)
0928 14:43:12.939828 (+ 1655us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:12.939833 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:12.939847 (+ 14us) server_negotiation.cc:407] Connection header received
0928 14:43:12.939895 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:12.939899 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:12.939921 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:12.940003 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:43:12.940239 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.941111 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.942565 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.943269 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.943340 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.943412 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:12.943765 (+ 353us) server_negotiation.cc:299] Negotiation successful
0928 14:43:12.943814 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1615,"thread_start_us":91,"threads_started":1}
I20260928 14:43:12.943984 16353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96e6b7dd4eda4bd9ab0229a111233f58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:12.944114 16717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:12.937636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:34405 (local address 127.0.0.1:57490)
0928 14:43:12.937943 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:12.937954 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:12.938008 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:12.940075 (+ 2067us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:12.940076 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:12.940077 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:43:12.940157 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.940159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.941215 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.941216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.942500 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:12.942503 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:12.943451 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:12.943452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:12.943530 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:12.943534 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:43:12.943584 (+ 50us) client_negotiation.cc:240] Negotiation successful
0928 14:43:12.944052 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224}
I20260928 14:43:12.944849 16490 tablet_service.cc:1467] Processing CreateTablet for tablet 96e6b7dd4eda4bd9ab0229a111233f58 (DEFAULT_TABLE table=test-table [id=a2a37e497696402093f9df5884633f0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:43:12.945089 16626 tablet_service.cc:1467] Processing CreateTablet for tablet 96e6b7dd4eda4bd9ab0229a111233f58 (DEFAULT_TABLE table=test-table [id=a2a37e497696402093f9df5884633f0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:43:12.945237 16490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96e6b7dd4eda4bd9ab0229a111233f58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:12.945397 16626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96e6b7dd4eda4bd9ab0229a111233f58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:12.946810 16722 tablet_bootstrap.cc:492] T 96e6b7dd4eda4bd9ab0229a111233f58 P dc15cdcdf4b145588845d6ce64504136: Bootstrap starting.
I20260928 14:43:12.947417 16722 tablet_bootstrap.cc:654] T 96e6b7dd4eda4bd9ab0229a111233f58 P dc15cdcdf4b145588845d6ce64504136: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:12.947634 16722 log.cc:824] T 96e6b7dd4eda4bd9ab0229a111233f58 P dc15cdcdf4b145588845d6ce64504136: Log is configured to *not* fsync() on all Append() calls
I20260928 14:43:12.948364 16722 tablet_bootstrap.cc:492] T 96e6b7dd4eda4bd9ab0229a111233f58 P dc15cdcdf4b145588845d6ce64504136: No bootstrap required, opened a new log
I20260928 14:43:12.948411 16722 ts_tablet_manager.cc:1397] T 96e6b7dd4eda4bd9ab0229a111233f58 P dc15cdcdf4b145588845d6ce64504136: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:43:12.948773 16724 tablet_bootstrap.cc:492] T 96e6b7dd4eda4bd9ab0229a111233f58 P b5fdff97460e49f5acef24e0105aab26: Bootstrap starting.
I20260928 14:43:12.949469 16724 tablet_bootstrap.cc:654] T 96e6b7dd4eda4bd9ab0229a111233f58 P b5fdff97460e49f5acef24e0105aab26: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:12.949802 16724 log.cc:824] T 96e6b7dd4eda4bd9ab0229a111233f58 P b5fdff97460e49f5acef24e0105aab26: Log is configured to *not* fsync() on all Append() calls
I20260928 14:43:12.950645 16724 tablet_bootstrap.cc:492] T 96e6b7dd4eda4bd9ab0229a111233f58 P b5fdff97460e49f5acef24e0105aab26: No bootstrap required, opened a new log
I20260928 14:43:12.950730 16724 ts_tablet_manager.cc:1397] T 96e6b7dd4eda4bd9ab0229a111233f58 P b5fdff97460e49f5acef24e0105aab26: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:43:12.951436 16722 raft_consensus.cc:348] T 96e6b7dd4eda4bd9ab0229a111233f58 P dc15cdcdf4b145588845d6ce64504136 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71899b6c516945508de4a4770b126e5f" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34405 } } peers { permanent_uuid: "b5fdff97460e49f5acef24e0105aab26" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 37471 } } peers { permanent_uuid: "dc15cdcdf4b145588845d6ce64504136" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 39055 } }
I20260928 14:43:12.951618 16722 raft_consensus.cc:374] T 96e6b7dd4eda4bd9ab0229a111233f58 P dc15cdcdf4b145588845d6ce64504136 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:12.951658 16722 raft_consensus.cc:729] T 96e6b7dd4eda4bd9ab0229a111233f58 P dc15cdcdf4b145588845d6ce64504136 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc15cdcdf4b145588845d6ce64504136, State: Initialized, Role: FOLLOWER
I20260928 14:43:12.951835 16722 consensus_queue.cc:260] T 96e6b7dd4eda4bd9ab0229a111233f58 P dc15cdcdf4b145588845d6ce64504136 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71899b6c516945508de4a4770b126e5f" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34405 } } peers { permanent_uuid: "b5fdff97460e49f5acef24e0105aab26" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 37471 } } peers { permanent_uuid: "dc15cdcdf4b145588845d6ce64504136" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 39055 } }
I20260928 14:43:12.951901 16725 tablet_bootstrap.cc:492] T 96e6b7dd4eda4bd9ab0229a111233f58 P 71899b6c516945508de4a4770b126e5f: Bootstrap starting.
I20260928 14:43:12.952111 16722 ts_tablet_manager.cc:1428] T 96e6b7dd4eda4bd9ab0229a111233f58 P dc15cdcdf4b145588845d6ce64504136: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260928 14:43:12.952246 16419 heartbeater.cc:503] Master 127.9.33.126:38209 was elected leader, sending a full tablet report...
I20260928 14:43:12.952602 16725 tablet_bootstrap.cc:654] T 96e6b7dd4eda4bd9ab0229a111233f58 P 71899b6c516945508de4a4770b126e5f: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:12.952910 16725 log.cc:824] T 96e6b7dd4eda4bd9ab0229a111233f58 P 71899b6c516945508de4a4770b126e5f: Log is configured to *not* fsync() on all Append() calls
I20260928 14:43:12.953469 16724 raft_consensus.cc:348] T 96e6b7dd4eda4bd9ab0229a111233f58 P b5fdff97460e49f5acef24e0105aab26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71899b6c516945508de4a4770b126e5f" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34405 } } peers { permanent_uuid: "b5fdff97460e49f5acef24e0105aab26" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 37471 } } peers { permanent_uuid: "dc15cdcdf4b145588845d6ce64504136" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 39055 } }
I20260928 14:43:12.953636 16724 raft_consensus.cc:374] T 96e6b7dd4eda4bd9ab0229a111233f58 P b5fdff97460e49f5acef24e0105aab26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:12.953688 16724 raft_consensus.cc:729] T 96e6b7dd4eda4bd9ab0229a111233f58 P b5fdff97460e49f5acef24e0105aab26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5fdff97460e49f5acef24e0105aab26, State: Initialized, Role: FOLLOWER
I20260928 14:43:12.953825 16724 consensus_queue.cc:260] T 96e6b7dd4eda4bd9ab0229a111233f58 P b5fdff97460e49f5acef24e0105aab26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71899b6c516945508de4a4770b126e5f" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34405 } } peers { permanent_uuid: "b5fdff97460e49f5acef24e0105aab26" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 37471 } } peers { permanent_uuid: "dc15cdcdf4b145588845d6ce64504136" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 39055 } }
I20260928 14:43:12.954035 16725 tablet_bootstrap.cc:492] T 96e6b7dd4eda4bd9ab0229a111233f58 P 71899b6c516945508de4a4770b126e5f: No bootstrap required, opened a new log
I20260928 14:43:12.954129 16725 ts_tablet_manager.cc:1397] T 96e6b7dd4eda4bd9ab0229a111233f58 P 71899b6c516945508de4a4770b126e5f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:43:12.954140 16724 ts_tablet_manager.cc:1428] T 96e6b7dd4eda4bd9ab0229a111233f58 P b5fdff97460e49f5acef24e0105aab26: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:43:12.954147 16556 heartbeater.cc:503] Master 127.9.33.126:38209 was elected leader, sending a full tablet report...
I20260928 14:43:12.956574 16725 raft_consensus.cc:348] T 96e6b7dd4eda4bd9ab0229a111233f58 P 71899b6c516945508de4a4770b126e5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71899b6c516945508de4a4770b126e5f" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34405 } } peers { permanent_uuid: "b5fdff97460e49f5acef24e0105aab26" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 37471 } } peers { permanent_uuid: "dc15cdcdf4b145588845d6ce64504136" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 39055 } }
I20260928 14:43:12.956732 16725 raft_consensus.cc:374] T 96e6b7dd4eda4bd9ab0229a111233f58 P 71899b6c516945508de4a4770b126e5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:12.956779 16725 raft_consensus.cc:729] T 96e6b7dd4eda4bd9ab0229a111233f58 P 71899b6c516945508de4a4770b126e5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71899b6c516945508de4a4770b126e5f, State: Initialized, Role: FOLLOWER
I20260928 14:43:12.956897 16725 consensus_queue.cc:260] T 96e6b7dd4eda4bd9ab0229a111233f58 P 71899b6c516945508de4a4770b126e5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71899b6c516945508de4a4770b126e5f" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34405 } } peers { permanent_uuid: "b5fdff97460e49f5acef24e0105aab26" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 37471 } } peers { permanent_uuid: "dc15cdcdf4b145588845d6ce64504136" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 39055 } }
I20260928 14:43:12.957185 16725 ts_tablet_manager.cc:1428] T 96e6b7dd4eda4bd9ab0229a111233f58 P 71899b6c516945508de4a4770b126e5f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:43:12.957257 16692 heartbeater.cc:503] Master 127.9.33.126:38209 was elected leader, sending a full tablet report...
I20260928 14:43:13.004173 16727 raft_consensus.cc:482] T 96e6b7dd4eda4bd9ab0229a111233f58 P dc15cdcdf4b145588845d6ce64504136 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:43:13.004340 16727 raft_consensus.cc:504] T 96e6b7dd4eda4bd9ab0229a111233f58 P dc15cdcdf4b145588845d6ce64504136 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71899b6c516945508de4a4770b126e5f" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34405 } } peers { permanent_uuid: "b5fdff97460e49f5acef24e0105aab26" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 37471 } } peers { permanent_uuid: "dc15cdcdf4b145588845d6ce64504136" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 39055 } }
I20260928 14:43:13.004747 16727 leader_election.cc:288] T 96e6b7dd4eda4bd9ab0229a111233f58 P dc15cdcdf4b145588845d6ce64504136 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71899b6c516945508de4a4770b126e5f (127.9.33.67:34405), b5fdff97460e49f5acef24e0105aab26 (127.9.33.66:37471)
I20260928 14:43:13.007687 16720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:13.004998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:48447 (local address 127.9.33.67:34405)
0928 14:43:13.005142 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:13.005144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:13.005151 (+ 7us) server_negotiation.cc:407] Connection header received
0928 14:43:13.005185 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:13.005186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:13.005195 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:13.005261 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:43:13.005423 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:13.005950 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:13.006955 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:13.007313 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:13.007406 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:13.007417 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:13.007573 (+ 156us) server_negotiation.cc:299] Negotiation successful
0928 14:43:13.007611 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260928 14:43:13.007705 16718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:13.005013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:57885 (local address 127.9.33.66:37471)
0928 14:43:13.005096 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:13.005098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:13.005178 (+ 80us) server_negotiation.cc:407] Connection header received
0928 14:43:13.005253 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:13.005254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:13.005267 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:13.005333 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:43:13.005491 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:13.005952 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:13.006958 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:13.007334 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:13.007418 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:13.007428 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:13.007579 (+ 151us) server_negotiation.cc:299] Negotiation successful
0928 14:43:13.007619 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260928 14:43:13.007702 16424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:13.004959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:37471 (local address 127.9.33.65:57885)
0928 14:43:13.005106 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:13.005117 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:13.005196 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:13.005349 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:13.005350 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:13.005351 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:43:13.005417 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:13.005418 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:13.006035 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:13.006037 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:13.006863 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:13.006865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:13.007417 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:13.007418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:13.007503 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:13.007506 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:43:13.007574 (+ 68us) client_negotiation.cc:240] Negotiation successful
0928 14:43:13.007616 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":138}
I20260928 14:43:13.008104 16510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96e6b7dd4eda4bd9ab0229a111233f58" candidate_uuid: "dc15cdcdf4b145588845d6ce64504136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5fdff97460e49f5acef24e0105aab26" is_pre_election: true
I20260928 14:43:13.007714 16425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:13.004939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:34405 (local address 127.9.33.65:48447)
0928 14:43:13.005018 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:13.005031 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:13.005082 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:13.005264 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:13.005265 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:13.005267 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:43:13.005352 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:13.005354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:13.006040 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:13.006042 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:13.006862 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:13.006864 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:13.007411 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:13.007412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:13.007487 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:13.007490 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:43:13.007565 (+ 75us) client_negotiation.cc:240] Negotiation successful
0928 14:43:13.007606 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":22}
I20260928 14:43:13.008277 16510 raft_consensus.cc:2454] T 96e6b7dd4eda4bd9ab0229a111233f58 P b5fdff97460e49f5acef24e0105aab26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc15cdcdf4b145588845d6ce64504136 in term 0.
I20260928 14:43:13.008554 16306 leader_election.cc:302] T 96e6b7dd4eda4bd9ab0229a111233f58 P dc15cdcdf4b145588845d6ce64504136 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fdff97460e49f5acef24e0105aab26, dc15cdcdf4b145588845d6ce64504136; no voters:
I20260928 14:43:13.008620 16646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96e6b7dd4eda4bd9ab0229a111233f58" candidate_uuid: "dc15cdcdf4b145588845d6ce64504136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71899b6c516945508de4a4770b126e5f" is_pre_election: true
I20260928 14:43:13.008718 16727 raft_consensus.cc:2781] T 96e6b7dd4eda4bd9ab0229a111233f58 P dc15cdcdf4b145588845d6ce64504136 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:43:13.008770 16646 raft_consensus.cc:2454] T 96e6b7dd4eda4bd9ab0229a111233f58 P 71899b6c516945508de4a4770b126e5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc15cdcdf4b145588845d6ce64504136 in term 0.
I20260928 14:43:13.008810 16727 raft_consensus.cc:482] T 96e6b7dd4eda4bd9ab0229a111233f58 P dc15cdcdf4b145588845d6ce64504136 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:43:13.008852 16727 raft_consensus.cc:3037] T 96e6b7dd4eda4bd9ab0229a111233f58 P dc15cdcdf4b145588845d6ce64504136 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:13.009648 16727 raft_consensus.cc:504] T 96e6b7dd4eda4bd9ab0229a111233f58 P dc15cdcdf4b145588845d6ce64504136 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71899b6c516945508de4a4770b126e5f" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34405 } } peers { permanent_uuid: "b5fdff97460e49f5acef24e0105aab26" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 37471 } } peers { permanent_uuid: "dc15cdcdf4b145588845d6ce64504136" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 39055 } }
I20260928 14:43:13.009845 16727 leader_election.cc:288] T 96e6b7dd4eda4bd9ab0229a111233f58 P dc15cdcdf4b145588845d6ce64504136 [CANDIDATE]: Term 1 election: Requested vote from peers 71899b6c516945508de4a4770b126e5f (127.9.33.67:34405), b5fdff97460e49f5acef24e0105aab26 (127.9.33.66:37471)
I20260928 14:43:13.010010 16646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96e6b7dd4eda4bd9ab0229a111233f58" candidate_uuid: "dc15cdcdf4b145588845d6ce64504136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71899b6c516945508de4a4770b126e5f"
I20260928 14:43:13.010089 16646 raft_consensus.cc:3037] T 96e6b7dd4eda4bd9ab0229a111233f58 P 71899b6c516945508de4a4770b126e5f [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:13.010054 16510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96e6b7dd4eda4bd9ab0229a111233f58" candidate_uuid: "dc15cdcdf4b145588845d6ce64504136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5fdff97460e49f5acef24e0105aab26"
I20260928 14:43:13.010175 16510 raft_consensus.cc:3037] T 96e6b7dd4eda4bd9ab0229a111233f58 P b5fdff97460e49f5acef24e0105aab26 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:13.010984 16510 raft_consensus.cc:2454] T 96e6b7dd4eda4bd9ab0229a111233f58 P b5fdff97460e49f5acef24e0105aab26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc15cdcdf4b145588845d6ce64504136 in term 1.
I20260928 14:43:13.011149 16306 leader_election.cc:302] T 96e6b7dd4eda4bd9ab0229a111233f58 P dc15cdcdf4b145588845d6ce64504136 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5fdff97460e49f5acef24e0105aab26, dc15cdcdf4b145588845d6ce64504136; no voters:
I20260928 14:43:13.011286 16727 raft_consensus.cc:2781] T 96e6b7dd4eda4bd9ab0229a111233f58 P dc15cdcdf4b145588845d6ce64504136 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:43:13.011404 16646 raft_consensus.cc:2454] T 96e6b7dd4eda4bd9ab0229a111233f58 P 71899b6c516945508de4a4770b126e5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc15cdcdf4b145588845d6ce64504136 in term 1.
I20260928 14:43:13.011431 16727 raft_consensus.cc:686] T 96e6b7dd4eda4bd9ab0229a111233f58 P dc15cdcdf4b145588845d6ce64504136 [term 1 LEADER]: Becoming Leader. State: Replica: dc15cdcdf4b145588845d6ce64504136, State: Running, Role: LEADER
I20260928 14:43:13.011566 16727 consensus_queue.cc:237] T 96e6b7dd4eda4bd9ab0229a111233f58 P dc15cdcdf4b145588845d6ce64504136 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71899b6c516945508de4a4770b126e5f" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34405 } } peers { permanent_uuid: "b5fdff97460e49f5acef24e0105aab26" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 37471 } } peers { permanent_uuid: "dc15cdcdf4b145588845d6ce64504136" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 39055 } }
I20260928 14:43:13.012811 15455 catalog_manager.cc:5495] T 96e6b7dd4eda4bd9ab0229a111233f58 P dc15cdcdf4b145588845d6ce64504136 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc15cdcdf4b145588845d6ce64504136 (127.9.33.65). New cstate: current_term: 1 leader_uuid: "dc15cdcdf4b145588845d6ce64504136" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71899b6c516945508de4a4770b126e5f" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5fdff97460e49f5acef24e0105aab26" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 37471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc15cdcdf4b145588845d6ce64504136" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 39055 } health_report { overall_health: HEALTHY } } }
W20260928 14:43:13.018406 16559 tablet.cc:2369] T 96e6b7dd4eda4bd9ab0229a111233f58 P b5fdff97460e49f5acef24e0105aab26: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:43:13.036268 16719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:13.033312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36376 (local address 127.9.33.65:39055)
0928 14:43:13.033378 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:13.033380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:13.033404 (+ 24us) server_negotiation.cc:407] Connection header received
0928 14:43:13.033442 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:13.033443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:13.033464 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:13.033708 (+ 244us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:43:13.033769 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:13.034365 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:13.035374 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:13.035569 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:13.035630 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:13.035863 (+ 233us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:43:13.036009 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:43:13.036078 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:13.036120 (+ 42us) server_negotiation.cc:299] Negotiation successful
0928 14:43:13.036141 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260928 14:43:13.038499 16510 raft_consensus.cc:1261] T 96e6b7dd4eda4bd9ab0229a111233f58 P b5fdff97460e49f5acef24e0105aab26 [term 1 FOLLOWER]: Refusing update from remote peer dc15cdcdf4b145588845d6ce64504136: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:43:13.038499 16646 raft_consensus.cc:1261] T 96e6b7dd4eda4bd9ab0229a111233f58 P 71899b6c516945508de4a4770b126e5f [term 1 FOLLOWER]: Refusing update from remote peer dc15cdcdf4b145588845d6ce64504136: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:43:13.038895 16727 consensus_queue.cc:1035] T 96e6b7dd4eda4bd9ab0229a111233f58 P dc15cdcdf4b145588845d6ce64504136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fdff97460e49f5acef24e0105aab26" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 37471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:43:13.039045 16731 consensus_queue.cc:1035] T 96e6b7dd4eda4bd9ab0229a111233f58 P dc15cdcdf4b145588845d6ce64504136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71899b6c516945508de4a4770b126e5f" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:43:13.042531 16735 mvcc.cc:205] Tried to move back new op lower bound from 7334324605081710592 to 7334324604976230400. Current Snapshot: MvccSnapshot[applied={T|T < 7334324605081710592}]
I20260928 14:43:13.042847 16736 mvcc.cc:205] Tried to move back new op lower bound from 7334324605081710592 to 7334324604976230400. Current Snapshot: MvccSnapshot[applied={T|T < 7334324605081710592}]
I20260928 14:43:13.044955 15452 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46316:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:43:13.045073 15452 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46316:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:43:13.046785 15452 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4cfad45886d34600a02df0ae697f86b2: Sending DeleteTablet for 3 replicas of tablet 96e6b7dd4eda4bd9ab0229a111233f58
I20260928 14:43:13.047176 16626 tablet_service.cc:1514] Processing DeleteTablet for tablet 96e6b7dd4eda4bd9ab0229a111233f58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:43:13 UTC) from {username='oryx', principal='oryx/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:57490
I20260928 14:43:13.047176 16490 tablet_service.cc:1514] Processing DeleteTablet for tablet 96e6b7dd4eda4bd9ab0229a111233f58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:43:13 UTC) from {username='oryx', principal='oryx/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:40204
I20260928 14:43:13.047211 16353 tablet_service.cc:1514] Processing DeleteTablet for tablet 96e6b7dd4eda4bd9ab0229a111233f58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:43:13 UTC) from {username='oryx', principal='oryx/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:36374
I20260928 14:43:13.047520 16743 tablet_replica.cc:330] T 96e6b7dd4eda4bd9ab0229a111233f58 P b5fdff97460e49f5acef24e0105aab26: stopping tablet replica
I20260928 14:43:13.047633 16744 tablet_replica.cc:330] T 96e6b7dd4eda4bd9ab0229a111233f58 P 71899b6c516945508de4a4770b126e5f: stopping tablet replica
I20260928 14:43:13.047653 16743 raft_consensus.cc:2229] T 96e6b7dd4eda4bd9ab0229a111233f58 P b5fdff97460e49f5acef24e0105aab26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:43:13.047807 16743 raft_consensus.cc:2258] T 96e6b7dd4eda4bd9ab0229a111233f58 P b5fdff97460e49f5acef24e0105aab26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:43:13.047837 16744 raft_consensus.cc:2229] T 96e6b7dd4eda4bd9ab0229a111233f58 P 71899b6c516945508de4a4770b126e5f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:43:13.048003 16745 tablet_replica.cc:330] T 96e6b7dd4eda4bd9ab0229a111233f58 P dc15cdcdf4b145588845d6ce64504136: stopping tablet replica
I20260928 14:43:13.048108 16745 raft_consensus.cc:2229] T 96e6b7dd4eda4bd9ab0229a111233f58 P dc15cdcdf4b145588845d6ce64504136 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:43:13.048110 16744 raft_consensus.cc:2258] T 96e6b7dd4eda4bd9ab0229a111233f58 P 71899b6c516945508de4a4770b126e5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:43:13.048269 16745 raft_consensus.cc:2258] T 96e6b7dd4eda4bd9ab0229a111233f58 P dc15cdcdf4b145588845d6ce64504136 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:43:13.048741 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 15423
I20260928 14:43:13.049199 16745 ts_tablet_manager.cc:1905] T 96e6b7dd4eda4bd9ab0229a111233f58 P dc15cdcdf4b145588845d6ce64504136: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:43:13.050304 16744 ts_tablet_manager.cc:1905] T 96e6b7dd4eda4bd9ab0229a111233f58 P 71899b6c516945508de4a4770b126e5f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:43:13.051432 16745 ts_tablet_manager.cc:1918] T 96e6b7dd4eda4bd9ab0229a111233f58 P dc15cdcdf4b145588845d6ce64504136: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:43:13.051530 16745 log.cc:1196] T 96e6b7dd4eda4bd9ab0229a111233f58 P dc15cdcdf4b145588845d6ce64504136: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-0/wal/wals/96e6b7dd4eda4bd9ab0229a111233f58
I20260928 14:43:13.052485 16744 ts_tablet_manager.cc:1918] T 96e6b7dd4eda4bd9ab0229a111233f58 P 71899b6c516945508de4a4770b126e5f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:43:13.052659 16744 log.cc:1196] T 96e6b7dd4eda4bd9ab0229a111233f58 P 71899b6c516945508de4a4770b126e5f: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-2/wal/wals/96e6b7dd4eda4bd9ab0229a111233f58
I20260928 14:43:13.053013 16745 ts_tablet_manager.cc:1939] T 96e6b7dd4eda4bd9ab0229a111233f58 P dc15cdcdf4b145588845d6ce64504136: Deleting consensus metadata
I20260928 14:43:13.053036 16744 ts_tablet_manager.cc:1939] T 96e6b7dd4eda4bd9ab0229a111233f58 P 71899b6c516945508de4a4770b126e5f: Deleting consensus metadata
W20260928 14:43:13.055948 16306 connection.cc:535] server connection from 127.0.0.1:36374 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260928 14:43:13.055990 16581 connection.cc:535] server connection from 127.0.0.1:57490 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260928 14:43:13.056524 16743 ts_tablet_manager.cc:1905] T 96e6b7dd4eda4bd9ab0229a111233f58 P b5fdff97460e49f5acef24e0105aab26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:43:13.059585 16743 ts_tablet_manager.cc:1918] T 96e6b7dd4eda4bd9ab0229a111233f58 P b5fdff97460e49f5acef24e0105aab26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:43:13.059764 16743 log.cc:1196] T 96e6b7dd4eda4bd9ab0229a111233f58 P b5fdff97460e49f5acef24e0105aab26: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-1/wal/wals/96e6b7dd4eda4bd9ab0229a111233f58
I20260928 14:43:13.060197 16743 ts_tablet_manager.cc:1939] T 96e6b7dd4eda4bd9ab0229a111233f58 P b5fdff97460e49f5acef24e0105aab26: Deleting consensus metadata
W20260928 14:43:13.060843 16443 connection.cc:441] server connection from 127.0.0.1:40204 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40204 (request call id 1) could send its response
I20260928 14:43:13.061157 16709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:13.048158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51046 (local address 127.9.33.124:40843)
0928 14:43:13.048230 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:13.048233 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:13.048608 (+ 375us) server_negotiation.cc:407] Connection header received
0928 14:43:13.051915 (+ 3307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:13.051916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:13.051932 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:13.051966 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:13.053549 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:13.054117 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:13.054987 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:13.055274 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:13.055342 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:13.057056 (+ 1714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:13.057059 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:13.057059 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:13.057068 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:13.059394 (+ 2326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:13.059846 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:13.059847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:13.059848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:13.059914 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:13.060153 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:13.060154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:13.060155 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:13.060326 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:13.060436 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:13.060990 (+ 554us) server_negotiation.cc:299] Negotiation successful
0928 14:43:13.061088 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260928 14:43:13.068651 16706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:13.059988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51060 (local address 127.9.33.124:40843)
0928 14:43:13.060032 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:13.060034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:13.060531 (+ 497us) server_negotiation.cc:407] Connection header received
0928 14:43:13.060625 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:13.060625 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:13.060638 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:13.060669 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:13.061572 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:13.062359 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:13.063442 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:13.063833 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:13.063906 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:13.065427 (+ 1521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:13.065428 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:13.065428 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:13.065436 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:13.067151 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:13.067568 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:13.067569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:13.067569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:13.067629 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:13.067889 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:13.067890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:13.067891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:13.068072 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:13.068161 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:13.068411 (+ 250us) server_negotiation.cc:299] Negotiation successful
0928 14:43:13.068510 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 14:43:13.069118 16707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:13.060173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47146 (local address 127.9.33.125:36321)
0928 14:43:13.060212 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:13.060214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:13.061460 (+ 1246us) server_negotiation.cc:407] Connection header received
0928 14:43:13.061547 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:13.061549 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:13.061567 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:13.061628 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:13.062328 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:13.062865 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:13.063825 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:13.064147 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:13.064216 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:13.065473 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:13.065475 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:13.065475 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:13.065482 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:13.067396 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:13.067920 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:13.067921 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:13.067922 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:13.067976 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:13.068190 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:13.068191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:13.068191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:13.068358 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:13.068449 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:13.068653 (+ 204us) server_negotiation.cc:299] Negotiation successful
0928 14:43:13.069019 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 14:43:14.045607 16754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.044696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38209 (local address 127.9.33.67:36085)
0928 14:43:14.045442 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:14.045532 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":720,"thread_start_us":73,"threads_started":1}
I20260928 14:43:14.046743 16755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.046428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38209 (local address 127.9.33.66:48433)
0928 14:43:14.046620 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:14.046690 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":59,"threads_started":1}
W20260928 14:43:14.047190 16556 heartbeater.cc:644] Failed to heartbeat to 127.9.33.126:38209 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.33.126:38209: connect: Connection refused (error 111)
I20260928 14:43:14.047976 16756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.047362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38209 (local address 127.9.33.65:50173)
0928 14:43:14.047846 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:14.047931 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":70,"threads_started":1}
W20260928 14:43:14.048075 16692 heartbeater.cc:644] Failed to heartbeat to 127.9.33.126:38209 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.33.126:38209: connect: Connection refused (error 111)
W20260928 14:43:14.048429 16419 heartbeater.cc:644] Failed to heartbeat to 127.9.33.126:38209 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.33.126:38209: connect: Connection refused (error 111)
I20260928 14:43:14.547430 16757 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4cfad45886d34600a02df0ae697f86b2)
I20260928 14:43:14.547498 16758 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4cfad45886d34600a02df0ae697f86b2)
I20260928 14:43:14.547636 16757 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: false } } peers { permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 40843 } attrs { promote: false } }
I20260928 14:43:14.547664 16758 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: false } } peers { permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 40843 } attrs { promote: false } }
I20260928 14:43:14.547977 16757 leader_election.cc:288] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4cfad45886d34600a02df0ae697f86b2 (127.9.33.126:38209), 39a978772cff40a6882ec08ae1ae2f52 (127.9.33.125:36321)
I20260928 14:43:14.548440 16759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.548137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38209 (local address 127.0.0.1:46806)
0928 14:43:14.548285 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:14.548348 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":49,"threads_started":1}
I20260928 14:43:14.548445 16758 leader_election.cc:288] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4cfad45886d34600a02df0ae697f86b2 (127.9.33.126:38209), ba6dfd46bca84cc5a9551e578c07703d (127.9.33.124:40843)
I20260928 14:43:14.548600 16760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.548128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38209 (local address 127.0.0.1:46812)
0928 14:43:14.548531 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:14.548567 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":60,"threads_started":1}
W20260928 14:43:14.548702 16224 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.33.126:38209: connect: Connection refused (error 111)
W20260928 14:43:14.548719 16038 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.33.126:38209: connect: Connection refused (error 111)
I20260928 14:43:14.549140 16760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.549053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38209 (local address 127.0.0.1:46826)
0928 14:43:14.549086 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:14.549111 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260928 14:43:14.549173 16759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.549085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38209 (local address 127.0.0.1:46832)
0928 14:43:14.549123 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:14.549147 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260928 14:43:14.549234 16224 leader_election.cc:334] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4cfad45886d34600a02df0ae697f86b2 (127.9.33.126:38209): Network error: Client connection negotiation failed: client connection to 127.9.33.126:38209: connect: Connection refused (error 111)
W20260928 14:43:14.549244 16038 leader_election.cc:334] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4cfad45886d34600a02df0ae697f86b2 (127.9.33.126:38209): Network error: Client connection negotiation failed: client connection to 127.9.33.126:38209: connect: Connection refused (error 111)
I20260928 14:43:14.552143 16762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.548332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:40843 (local address 127.0.0.1:51070)
0928 14:43:14.548537 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:14.548547 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:14.548594 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:14.548865 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:14.548866 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:14.548869 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:43:14.548936 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:14.548938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:14.549881 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:14.549882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:14.551150 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:14.551153 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:14.551872 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:14.551873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:14.551945 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:14.551950 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:43:14.552005 (+ 55us) client_negotiation.cc:240] Negotiation successful
0928 14:43:14.552052 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":43,"threads_started":1}
I20260928 14:43:14.552136 16764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.548445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51070 (local address 127.9.33.124:40843)
0928 14:43:14.548698 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:14.548700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:14.548707 (+ 7us) server_negotiation.cc:407] Connection header received
0928 14:43:14.548734 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:14.548735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:14.548745 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:14.548809 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:43:14.549018 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:14.549804 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:14.551234 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:14.551785 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:14.551867 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:14.551882 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:14.552026 (+ 144us) server_negotiation.cc:299] Negotiation successful
0928 14:43:14.552065 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":41,"threads_started":1}
I20260928 14:43:14.552428 16250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "39a978772cff40a6882ec08ae1ae2f52" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ba6dfd46bca84cc5a9551e578c07703d" is_pre_election: true
I20260928 14:43:14.552533 16250 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39a978772cff40a6882ec08ae1ae2f52 in term 1.
I20260928 14:43:14.552683 16038 leader_election.cc:302] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39a978772cff40a6882ec08ae1ae2f52, ba6dfd46bca84cc5a9551e578c07703d; no voters: 4cfad45886d34600a02df0ae697f86b2
I20260928 14:43:14.552783 16758 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260928 14:43:14.552853 16758 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4cfad45886d34600a02df0ae697f86b2)
I20260928 14:43:14.552886 16758 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [term 1 FOLLOWER]: Advancing to term 2
I20260928 14:43:14.553057 16761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.548155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.125:36321 (local address 127.0.0.1:47158)
0928 14:43:14.548386 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:14.548398 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:14.548459 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:14.549058 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:14.549059 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:14.549062 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:43:14.549117 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:14.549119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:14.550156 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:14.550157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:14.551066 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:14.551068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:14.552799 (+ 1731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:14.552800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:14.552893 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:14.552897 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:43:14.552947 (+ 50us) client_negotiation.cc:240] Negotiation successful
0928 14:43:14.552992 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":3,"thread_start_us":41,"threads_started":1}
I20260928 14:43:14.553283 16763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.548395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47158 (local address 127.9.33.125:36321)
0928 14:43:14.548717 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:14.548719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:14.548729 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:43:14.548755 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:14.548756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:14.548873 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:14.549015 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:43:14.549382 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:14.550066 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:14.551173 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:14.552112 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:14.552175 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:14.552186 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:14.553161 (+ 975us) server_negotiation.cc:299] Negotiation successful
0928 14:43:14.553212 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":32,"threads_started":1}
I20260928 14:43:14.553471 16064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ba6dfd46bca84cc5a9551e578c07703d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "39a978772cff40a6882ec08ae1ae2f52" is_pre_election: true
I20260928 14:43:14.553836 16758 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: false } } peers { permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 40843 } attrs { promote: false } }
I20260928 14:43:14.553989 16758 leader_election.cc:288] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [CANDIDATE]: Term 2 election: Requested vote from peers 4cfad45886d34600a02df0ae697f86b2 (127.9.33.126:38209), ba6dfd46bca84cc5a9551e578c07703d (127.9.33.124:40843)
I20260928 14:43:14.554020 16064 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ba6dfd46bca84cc5a9551e578c07703d in current term 2: Already voted for candidate 39a978772cff40a6882ec08ae1ae2f52 in this term.
I20260928 14:43:14.554167 16762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.554081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38209 (local address 127.0.0.1:46860)
0928 14:43:14.554114 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:14.554151 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260928 14:43:14.554255 16226 leader_election.cc:302] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba6dfd46bca84cc5a9551e578c07703d; no voters: 39a978772cff40a6882ec08ae1ae2f52, 4cfad45886d34600a02df0ae697f86b2
I20260928 14:43:14.554239 16250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "39a978772cff40a6882ec08ae1ae2f52" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ba6dfd46bca84cc5a9551e578c07703d"
I20260928 14:43:14.554379 16757 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d [term 1 FOLLOWER]: Advancing to term 2
I20260928 14:43:14.554498 16762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.554416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38209 (local address 127.0.0.1:46870)
0928 14:43:14.554451 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:14.554471 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260928 14:43:14.554549 16038 leader_election.cc:334] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 4cfad45886d34600a02df0ae697f86b2 (127.9.33.126:38209): Network error: Client connection negotiation failed: client connection to 127.9.33.126:38209: connect: Connection refused (error 111)
I20260928 14:43:14.555172 16757 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 14:43:14.555990 16250 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39a978772cff40a6882ec08ae1ae2f52 in term 2.
I20260928 14:43:14.556439 16038 leader_election.cc:302] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 39a978772cff40a6882ec08ae1ae2f52, ba6dfd46bca84cc5a9551e578c07703d; no voters: 4cfad45886d34600a02df0ae697f86b2
I20260928 14:43:14.556555 16758 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [term 2 FOLLOWER]: Leader election won for term 2
I20260928 14:43:14.556706 16758 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [term 2 LEADER]: Becoming Leader. State: Replica: 39a978772cff40a6882ec08ae1ae2f52, State: Running, Role: LEADER
I20260928 14:43:14.556795 16758 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: false } } peers { permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 40843 } attrs { promote: false } }
I20260928 14:43:14.557211 16767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39a978772cff40a6882ec08ae1ae2f52. Latest consensus state: current_term: 2 leader_uuid: "39a978772cff40a6882ec08ae1ae2f52" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: false } } peers { permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 40843 } attrs { promote: false } } }
I20260928 14:43:14.557343 16767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [sys.catalog]: This master's current role is: LEADER
I20260928 14:43:14.557545 16769 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:43:14.557936 16769 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:43:14.558065 16769 catalog_manager.cc:1259] Loaded cluster ID: 063a364e0f4846ca91dc5cd374c69e34
I20260928 14:43:14.558107 16769 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:43:14.558280 16769 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:43:14.558339 16769 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52: Loaded TSK: 0
I20260928 14:43:14.558439 16769 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:43:14.614923 16054 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47146:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:43:14.616717 16054 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:43:14.617748 16762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.617638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38209 (local address 127.0.0.1:46886)
0928 14:43:14.617688 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:14.617721 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260928 14:43:14.617892 16250 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d [term 2 FOLLOWER]: Refusing update from remote peer 39a978772cff40a6882ec08ae1ae2f52: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260928 14:43:14.618094 16762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.618005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38209 (local address 127.0.0.1:46898)
0928 14:43:14.618041 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:14.618070 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260928 14:43:14.618127 16758 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 40843 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260928 14:43:14.618309 16038 consensus_peers.cc:487] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 -> Peer 4cfad45886d34600a02df0ae697f86b2 (127.9.33.126:38209): Couldn't send request to peer 4cfad45886d34600a02df0ae697f86b2. Status: Network error: Client connection negotiation failed: client connection to 127.9.33.126:38209: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260928 14:43:14.618844 16757 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39a978772cff40a6882ec08ae1ae2f52. Latest consensus state: current_term: 2 leader_uuid: "39a978772cff40a6882ec08ae1ae2f52" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: false } } peers { permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 40843 } attrs { promote: false } } }
I20260928 14:43:14.618928 16757 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:43:14.619380 16770 mvcc.cc:205] Tried to move back new op lower bound from 7334324611552157696 to 7334324611305345024. Current Snapshot: MvccSnapshot[applied={T|T < 7334324611552157696}]
I20260928 14:43:14.619462 16767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "39a978772cff40a6882ec08ae1ae2f52" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: false } } peers { permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 40843 } attrs { promote: false } } }
I20260928 14:43:14.619547 16767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52 [sys.catalog]: This master's current role is: LEADER
I20260928 14:43:14.619576 16757 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "39a978772cff40a6882ec08ae1ae2f52" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "4cfad45886d34600a02df0ae697f86b2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38209 } } peers { permanent_uuid: "39a978772cff40a6882ec08ae1ae2f52" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36321 } attrs { promote: false } } peers { permanent_uuid: "ba6dfd46bca84cc5a9551e578c07703d" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 40843 } attrs { promote: false } } }
I20260928 14:43:14.619668 16757 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba6dfd46bca84cc5a9551e578c07703d [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:43:14.619776 16771 mvcc.cc:205] Tried to move back new op lower bound from 7334324611552157696 to 7334324611305345024. Current Snapshot: MvccSnapshot[applied={T|T < 7334324611552157696 or (T in {7334324611552157696})}]
I20260928 14:43:14.625222 16779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.621875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57492 (local address 127.9.33.67:34405)
0928 14:43:14.621986 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:14.621989 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:14.621998 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:43:14.622025 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:14.622026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:14.622037 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:14.622102 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:43:14.622270 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:14.623083 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:14.624234 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:14.624840 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:14.624957 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:14.624973 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:14.625104 (+ 131us) server_negotiation.cc:299] Negotiation successful
0928 14:43:14.625143 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":56,"threads_started":1}
I20260928 14:43:14.625236 16762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.621562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:37471 (local address 127.0.0.1:40216)
0928 14:43:14.621621 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:14.621634 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:14.621700 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:14.622138 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:14.622138 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:14.622139 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:43:14.622281 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:14.622282 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:14.622998 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:14.622999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:14.624203 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:14.624205 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:14.624959 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:14.624960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:14.625028 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:14.625033 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:43:14.625091 (+ 58us) client_negotiation.cc:240] Negotiation successful
0928 14:43:14.625138 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":100}
I20260928 14:43:14.625237 16760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.621667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:34405 (local address 127.0.0.1:57492)
0928 14:43:14.621782 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:14.621793 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:14.621850 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:14.622120 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:14.622121 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:14.622124 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:43:14.622195 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:14.622197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:14.623192 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:14.623193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:14.624144 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:14.624147 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:14.624958 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:14.624959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:14.625031 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:14.625034 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:43:14.625083 (+ 49us) client_negotiation.cc:240] Negotiation successful
0928 14:43:14.625114 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":2,"thread_start_us":75,"threads_started":1}
I20260928 14:43:14.625466 16777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.621566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:39055 (local address 127.0.0.1:36390)
0928 14:43:14.621796 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:14.621805 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:14.621848 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:14.622141 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:14.622142 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:14.622143 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:43:14.622191 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:14.622192 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:14.623162 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:14.623163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:14.624290 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:14.624292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:14.624890 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:14.624891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:14.624956 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:14.624962 (+ 6us) client_negotiation.cc:769] Sending connection context
0928 14:43:14.625011 (+ 49us) client_negotiation.cc:240] Negotiation successful
0928 14:43:14.625066 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":49}
I20260928 14:43:14.625528 16776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.621658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40216 (local address 127.9.33.66:37471)
0928 14:43:14.621998 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:14.622000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:14.622008 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:43:14.622036 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:14.622036 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:14.622047 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:14.622170 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:43:14.622366 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:14.622896 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:14.624278 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:14.624858 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:14.625363 (+ 505us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:14.625383 (+ 20us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:14.625404 (+ 21us) server_negotiation.cc:299] Negotiation successful
0928 14:43:14.625453 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":68,"threads_started":1}
I20260928 14:43:14.625762 16490 tablet_service.cc:1467] Processing CreateTablet for tablet 8e2e78fe96fa4663a27988d639f24cea (DEFAULT_TABLE table=test-table [id=a9b1c7a6af134133a7fa309293033d2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:43:14.625780 16626 tablet_service.cc:1467] Processing CreateTablet for tablet 8e2e78fe96fa4663a27988d639f24cea (DEFAULT_TABLE table=test-table [id=a9b1c7a6af134133a7fa309293033d2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:43:14.625908 16626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e2e78fe96fa4663a27988d639f24cea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:14.625973 16490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e2e78fe96fa4663a27988d639f24cea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:14.626293 16778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.621765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36390 (local address 127.9.33.65:39055)
0928 14:43:14.621994 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:14.621996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:14.622004 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:43:14.622028 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:14.622028 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:14.622039 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:14.622281 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:43:14.622323 (+ 42us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:14.623082 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:14.624373 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:14.624782 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:14.624887 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:14.624901 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:14.625168 (+ 267us) server_negotiation.cc:299] Negotiation successful
0928 14:43:14.625221 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":65,"threads_started":1}
I20260928 14:43:14.626539 16353 tablet_service.cc:1467] Processing CreateTablet for tablet 8e2e78fe96fa4663a27988d639f24cea (DEFAULT_TABLE table=test-table [id=a9b1c7a6af134133a7fa309293033d2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:43:14.626677 16353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e2e78fe96fa4663a27988d639f24cea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:14.627619 16780 tablet_bootstrap.cc:492] T 8e2e78fe96fa4663a27988d639f24cea P b5fdff97460e49f5acef24e0105aab26: Bootstrap starting.
I20260928 14:43:14.628135 16780 tablet_bootstrap.cc:654] T 8e2e78fe96fa4663a27988d639f24cea P b5fdff97460e49f5acef24e0105aab26: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:14.628264 16782 tablet_bootstrap.cc:492] T 8e2e78fe96fa4663a27988d639f24cea P dc15cdcdf4b145588845d6ce64504136: Bootstrap starting.
I20260928 14:43:14.628479 16781 tablet_bootstrap.cc:492] T 8e2e78fe96fa4663a27988d639f24cea P 71899b6c516945508de4a4770b126e5f: Bootstrap starting.
I20260928 14:43:14.628687 16782 tablet_bootstrap.cc:654] T 8e2e78fe96fa4663a27988d639f24cea P dc15cdcdf4b145588845d6ce64504136: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:14.628866 16781 tablet_bootstrap.cc:654] T 8e2e78fe96fa4663a27988d639f24cea P 71899b6c516945508de4a4770b126e5f: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:14.629320 16780 tablet_bootstrap.cc:492] T 8e2e78fe96fa4663a27988d639f24cea P b5fdff97460e49f5acef24e0105aab26: No bootstrap required, opened a new log
I20260928 14:43:14.629352 16782 tablet_bootstrap.cc:492] T 8e2e78fe96fa4663a27988d639f24cea P dc15cdcdf4b145588845d6ce64504136: No bootstrap required, opened a new log
I20260928 14:43:14.629379 16780 ts_tablet_manager.cc:1397] T 8e2e78fe96fa4663a27988d639f24cea P b5fdff97460e49f5acef24e0105aab26: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:43:14.629413 16782 ts_tablet_manager.cc:1397] T 8e2e78fe96fa4663a27988d639f24cea P dc15cdcdf4b145588845d6ce64504136: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:14.629500 16781 tablet_bootstrap.cc:492] T 8e2e78fe96fa4663a27988d639f24cea P 71899b6c516945508de4a4770b126e5f: No bootstrap required, opened a new log
I20260928 14:43:14.629550 16781 ts_tablet_manager.cc:1397] T 8e2e78fe96fa4663a27988d639f24cea P 71899b6c516945508de4a4770b126e5f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:14.629542 16782 raft_consensus.cc:348] T 8e2e78fe96fa4663a27988d639f24cea P dc15cdcdf4b145588845d6ce64504136 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5fdff97460e49f5acef24e0105aab26" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 37471 } } peers { permanent_uuid: "dc15cdcdf4b145588845d6ce64504136" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 39055 } } peers { permanent_uuid: "71899b6c516945508de4a4770b126e5f" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34405 } }
I20260928 14:43:14.629554 16780 raft_consensus.cc:348] T 8e2e78fe96fa4663a27988d639f24cea P b5fdff97460e49f5acef24e0105aab26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5fdff97460e49f5acef24e0105aab26" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 37471 } } peers { permanent_uuid: "dc15cdcdf4b145588845d6ce64504136" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 39055 } } peers { permanent_uuid: "71899b6c516945508de4a4770b126e5f" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34405 } }
I20260928 14:43:14.629630 16782 raft_consensus.cc:374] T 8e2e78fe96fa4663a27988d639f24cea P dc15cdcdf4b145588845d6ce64504136 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:14.629629 16780 raft_consensus.cc:374] T 8e2e78fe96fa4663a27988d639f24cea P b5fdff97460e49f5acef24e0105aab26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:14.629659 16780 raft_consensus.cc:729] T 8e2e78fe96fa4663a27988d639f24cea P b5fdff97460e49f5acef24e0105aab26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5fdff97460e49f5acef24e0105aab26, State: Initialized, Role: FOLLOWER
I20260928 14:43:14.629659 16782 raft_consensus.cc:729] T 8e2e78fe96fa4663a27988d639f24cea P dc15cdcdf4b145588845d6ce64504136 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc15cdcdf4b145588845d6ce64504136, State: Initialized, Role: FOLLOWER
I20260928 14:43:14.629660 16781 raft_consensus.cc:348] T 8e2e78fe96fa4663a27988d639f24cea P 71899b6c516945508de4a4770b126e5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5fdff97460e49f5acef24e0105aab26" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 37471 } } peers { permanent_uuid: "dc15cdcdf4b145588845d6ce64504136" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 39055 } } peers { permanent_uuid: "71899b6c516945508de4a4770b126e5f" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34405 } }
I20260928 14:43:14.629732 16781 raft_consensus.cc:374] T 8e2e78fe96fa4663a27988d639f24cea P 71899b6c516945508de4a4770b126e5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:14.629726 16780 consensus_queue.cc:260] T 8e2e78fe96fa4663a27988d639f24cea P b5fdff97460e49f5acef24e0105aab26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5fdff97460e49f5acef24e0105aab26" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 37471 } } peers { permanent_uuid: "dc15cdcdf4b145588845d6ce64504136" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 39055 } } peers { permanent_uuid: "71899b6c516945508de4a4770b126e5f" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34405 } }
I20260928 14:43:14.629761 16781 raft_consensus.cc:729] T 8e2e78fe96fa4663a27988d639f24cea P 71899b6c516945508de4a4770b126e5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71899b6c516945508de4a4770b126e5f, State: Initialized, Role: FOLLOWER
I20260928 14:43:14.629732 16782 consensus_queue.cc:260] T 8e2e78fe96fa4663a27988d639f24cea P dc15cdcdf4b145588845d6ce64504136 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5fdff97460e49f5acef24e0105aab26" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 37471 } } peers { permanent_uuid: "dc15cdcdf4b145588845d6ce64504136" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 39055 } } peers { permanent_uuid: "71899b6c516945508de4a4770b126e5f" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34405 } }
I20260928 14:43:14.629801 16781 consensus_queue.cc:260] T 8e2e78fe96fa4663a27988d639f24cea P 71899b6c516945508de4a4770b126e5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5fdff97460e49f5acef24e0105aab26" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 37471 } } peers { permanent_uuid: "dc15cdcdf4b145588845d6ce64504136" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 39055 } } peers { permanent_uuid: "71899b6c516945508de4a4770b126e5f" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34405 } }
I20260928 14:43:14.629894 16780 ts_tablet_manager.cc:1428] T 8e2e78fe96fa4663a27988d639f24cea P b5fdff97460e49f5acef24e0105aab26: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260928 14:43:14.629897 16782 ts_tablet_manager.cc:1428] T 8e2e78fe96fa4663a27988d639f24cea P dc15cdcdf4b145588845d6ce64504136: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260928 14:43:14.629959 16781 ts_tablet_manager.cc:1428] T 8e2e78fe96fa4663a27988d639f24cea P 71899b6c516945508de4a4770b126e5f: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260928 14:43:14.630386 16789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.630131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38209 (local address 127.9.33.65:60567)
0928 14:43:14.630303 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:14.630348 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":88,"threads_started":1}
I20260928 14:43:14.630414 16790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.630131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38209 (local address 127.9.33.66:39501)
0928 14:43:14.630335 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:14.630373 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":113,"threads_started":1}
I20260928 14:43:14.631145 16558 heartbeater.cc:503] Master 127.9.33.125:36321 was elected leader, sending a full tablet report...
I20260928 14:43:14.631157 16694 heartbeater.cc:503] Master 127.9.33.125:36321 was elected leader, sending a full tablet report...
I20260928 14:43:14.631484 16790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.631078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38209 (local address 127.9.33.66:56669)
0928 14:43:14.631293 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:14.631454 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197}
I20260928 14:43:14.631616 16421 heartbeater.cc:503] Master 127.9.33.125:36321 was elected leader, sending a full tablet report...
I20260928 14:43:14.631784 16791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.630186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38209 (local address 127.9.33.67:49745)
0928 14:43:14.631659 (+ 1473us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:14.631698 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1448,"thread_start_us":89,"threads_started":1}
I20260928 14:43:14.632268 16791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.632033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38209 (local address 127.9.33.67:43633)
0928 14:43:14.632216 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:14.632245 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168}
I20260928 14:43:14.632607 16789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.632315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38209 (local address 127.9.33.65:40745)
0928 14:43:14.632550 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:14.632577 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217}
W20260928 14:43:14.634565 16422 tablet.cc:2369] T 8e2e78fe96fa4663a27988d639f24cea P dc15cdcdf4b145588845d6ce64504136: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:43:14.661319 16695 tablet.cc:2369] T 8e2e78fe96fa4663a27988d639f24cea P 71899b6c516945508de4a4770b126e5f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:43:14.774286 16788 raft_consensus.cc:482] T 8e2e78fe96fa4663a27988d639f24cea P 71899b6c516945508de4a4770b126e5f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:43:14.774434 16788 raft_consensus.cc:504] T 8e2e78fe96fa4663a27988d639f24cea P 71899b6c516945508de4a4770b126e5f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5fdff97460e49f5acef24e0105aab26" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 37471 } } peers { permanent_uuid: "dc15cdcdf4b145588845d6ce64504136" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 39055 } } peers { permanent_uuid: "71899b6c516945508de4a4770b126e5f" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34405 } }
I20260928 14:43:14.774825 16788 leader_election.cc:288] T 8e2e78fe96fa4663a27988d639f24cea P 71899b6c516945508de4a4770b126e5f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5fdff97460e49f5acef24e0105aab26 (127.9.33.66:37471), dc15cdcdf4b145588845d6ce64504136 (127.9.33.65:39055)
I20260928 14:43:14.777796 16791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.775056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:39055 (local address 127.9.33.67:50825)
0928 14:43:14.775131 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:14.775145 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:14.775207 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:14.775339 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:14.775339 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:14.775341 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:43:14.775405 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:14.775409 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:14.776169 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:14.776170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:14.776967 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:14.776969 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:14.777534 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:14.777534 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:14.777605 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:14.777608 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:43:14.777675 (+ 67us) client_negotiation.cc:240] Negotiation successful
0928 14:43:14.777719 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260928 14:43:14.777796 16778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.775159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:50825 (local address 127.9.33.65:39055)
0928 14:43:14.775204 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:14.775206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:14.775216 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:43:14.775271 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:14.775272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:14.775285 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:14.775336 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:43:14.775496 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:14.776078 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:14.777072 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:14.777445 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:14.777532 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:14.777544 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:14.777682 (+ 138us) server_negotiation.cc:299] Negotiation successful
0928 14:43:14.777721 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 14:43:14.778062 16776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.775126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:56415 (local address 127.9.33.66:37471)
0928 14:43:14.775184 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:14.775186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:14.775409 (+ 223us) server_negotiation.cc:407] Connection header received
0928 14:43:14.775512 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:14.775514 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:14.775523 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:14.775592 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:43:14.775781 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:14.776360 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:14.777288 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:14.777650 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:14.777734 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:14.777746 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:14.777938 (+ 192us) server_negotiation.cc:299] Negotiation successful
0928 14:43:14.777998 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260928 14:43:14.778124 16373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e2e78fe96fa4663a27988d639f24cea" candidate_uuid: "71899b6c516945508de4a4770b126e5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc15cdcdf4b145588845d6ce64504136" is_pre_election: true
I20260928 14:43:14.778286 16373 raft_consensus.cc:2454] T 8e2e78fe96fa4663a27988d639f24cea P dc15cdcdf4b145588845d6ce64504136 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71899b6c516945508de4a4770b126e5f in term 0.
I20260928 14:43:14.778313 16792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.775056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:37471 (local address 127.9.33.67:56415)
0928 14:43:14.775328 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:14.775336 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:14.775430 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:14.775608 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:14.775609 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:14.775610 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:43:14.775675 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:14.775677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:14.776475 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:14.776476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:14.777199 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:14.777201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:14.777749 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:14.777751 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:14.777824 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:14.777827 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:43:14.778181 (+ 354us) client_negotiation.cc:240] Negotiation successful
0928 14:43:14.778228 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":53,"thread_start_us":158,"threads_started":1}
I20260928 14:43:14.778566 16580 leader_election.cc:302] T 8e2e78fe96fa4663a27988d639f24cea P 71899b6c516945508de4a4770b126e5f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71899b6c516945508de4a4770b126e5f, dc15cdcdf4b145588845d6ce64504136; no voters:
I20260928 14:43:14.778573 16510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e2e78fe96fa4663a27988d639f24cea" candidate_uuid: "71899b6c516945508de4a4770b126e5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5fdff97460e49f5acef24e0105aab26" is_pre_election: true
I20260928 14:43:14.778661 16510 raft_consensus.cc:2454] T 8e2e78fe96fa4663a27988d639f24cea P b5fdff97460e49f5acef24e0105aab26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71899b6c516945508de4a4770b126e5f in term 0.
I20260928 14:43:14.778733 16788 raft_consensus.cc:2781] T 8e2e78fe96fa4663a27988d639f24cea P 71899b6c516945508de4a4770b126e5f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:43:14.778800 16788 raft_consensus.cc:482] T 8e2e78fe96fa4663a27988d639f24cea P 71899b6c516945508de4a4770b126e5f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:43:14.778821 16788 raft_consensus.cc:3037] T 8e2e78fe96fa4663a27988d639f24cea P 71899b6c516945508de4a4770b126e5f [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:14.779569 16788 raft_consensus.cc:504] T 8e2e78fe96fa4663a27988d639f24cea P 71899b6c516945508de4a4770b126e5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5fdff97460e49f5acef24e0105aab26" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 37471 } } peers { permanent_uuid: "dc15cdcdf4b145588845d6ce64504136" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 39055 } } peers { permanent_uuid: "71899b6c516945508de4a4770b126e5f" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34405 } }
I20260928 14:43:14.779803 16788 leader_election.cc:288] T 8e2e78fe96fa4663a27988d639f24cea P 71899b6c516945508de4a4770b126e5f [CANDIDATE]: Term 1 election: Requested vote from peers b5fdff97460e49f5acef24e0105aab26 (127.9.33.66:37471), dc15cdcdf4b145588845d6ce64504136 (127.9.33.65:39055)
I20260928 14:43:14.779907 16510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e2e78fe96fa4663a27988d639f24cea" candidate_uuid: "71899b6c516945508de4a4770b126e5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5fdff97460e49f5acef24e0105aab26"
I20260928 14:43:14.779989 16510 raft_consensus.cc:3037] T 8e2e78fe96fa4663a27988d639f24cea P b5fdff97460e49f5acef24e0105aab26 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:14.779963 16373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e2e78fe96fa4663a27988d639f24cea" candidate_uuid: "71899b6c516945508de4a4770b126e5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc15cdcdf4b145588845d6ce64504136"
I20260928 14:43:14.780027 16373 raft_consensus.cc:3037] T 8e2e78fe96fa4663a27988d639f24cea P dc15cdcdf4b145588845d6ce64504136 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:14.780884 16510 raft_consensus.cc:2454] T 8e2e78fe96fa4663a27988d639f24cea P b5fdff97460e49f5acef24e0105aab26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71899b6c516945508de4a4770b126e5f in term 1.
I20260928 14:43:14.780884 16373 raft_consensus.cc:2454] T 8e2e78fe96fa4663a27988d639f24cea P dc15cdcdf4b145588845d6ce64504136 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71899b6c516945508de4a4770b126e5f in term 1.
I20260928 14:43:14.781116 16579 leader_election.cc:302] T 8e2e78fe96fa4663a27988d639f24cea P 71899b6c516945508de4a4770b126e5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71899b6c516945508de4a4770b126e5f, b5fdff97460e49f5acef24e0105aab26; no voters:
I20260928 14:43:14.781229 16788 raft_consensus.cc:2781] T 8e2e78fe96fa4663a27988d639f24cea P 71899b6c516945508de4a4770b126e5f [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:43:14.781426 16788 raft_consensus.cc:686] T 8e2e78fe96fa4663a27988d639f24cea P 71899b6c516945508de4a4770b126e5f [term 1 LEADER]: Becoming Leader. State: Replica: 71899b6c516945508de4a4770b126e5f, State: Running, Role: LEADER
I20260928 14:43:14.781523 16788 consensus_queue.cc:237] T 8e2e78fe96fa4663a27988d639f24cea P 71899b6c516945508de4a4770b126e5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5fdff97460e49f5acef24e0105aab26" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 37471 } } peers { permanent_uuid: "dc15cdcdf4b145588845d6ce64504136" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 39055 } } peers { permanent_uuid: "71899b6c516945508de4a4770b126e5f" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34405 } }
I20260928 14:43:14.781771 16792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.781666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38209 (local address 127.9.33.67:45495)
0928 14:43:14.781702 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:14.781739 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260928 14:43:14.782411 16053 catalog_manager.cc:5495] T 8e2e78fe96fa4663a27988d639f24cea P 71899b6c516945508de4a4770b126e5f reported cstate change: term changed from 0 to 1, leader changed from <none> to 71899b6c516945508de4a4770b126e5f (127.9.33.67). New cstate: current_term: 1 leader_uuid: "71899b6c516945508de4a4770b126e5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5fdff97460e49f5acef24e0105aab26" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 37471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc15cdcdf4b145588845d6ce64504136" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 39055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71899b6c516945508de4a4770b126e5f" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34405 } health_report { overall_health: HEALTHY } } }
I20260928 14:43:14.812994 16779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.810476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57508 (local address 127.9.33.67:34405)
0928 14:43:14.810542 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:14.810544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:14.810552 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:43:14.810588 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:14.810589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:14.810626 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:14.810718 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:43:14.810869 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:14.811353 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:14.812182 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:14.812384 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:14.812465 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:14.812641 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:43:14.812745 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:43:14.812842 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:14.812868 (+ 26us) server_negotiation.cc:299] Negotiation successful
0928 14:43:14.812894 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260928 14:43:14.814338 16510 raft_consensus.cc:1261] T 8e2e78fe96fa4663a27988d639f24cea P b5fdff97460e49f5acef24e0105aab26 [term 1 FOLLOWER]: Refusing update from remote peer 71899b6c516945508de4a4770b126e5f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:43:14.814417 16373 raft_consensus.cc:1261] T 8e2e78fe96fa4663a27988d639f24cea P dc15cdcdf4b145588845d6ce64504136 [term 1 FOLLOWER]: Refusing update from remote peer 71899b6c516945508de4a4770b126e5f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:43:14.814875 16788 consensus_queue.cc:1035] T 8e2e78fe96fa4663a27988d639f24cea P 71899b6c516945508de4a4770b126e5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc15cdcdf4b145588845d6ce64504136" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 39055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:43:14.815073 16793 consensus_queue.cc:1035] T 8e2e78fe96fa4663a27988d639f24cea P 71899b6c516945508de4a4770b126e5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5fdff97460e49f5acef24e0105aab26" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 37471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:43:14.816203 16790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.815962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38209 (local address 127.9.33.66:39663)
0928 14:43:14.816130 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:14.816164 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147}
I20260928 14:43:14.816241 16789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.816077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38209 (local address 127.9.33.65:44159)
0928 14:43:14.816160 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:14.816201 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260928 14:43:14.817687 16792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.817518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38209 (local address 127.9.33.67:38791)
0928 14:43:14.817615 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:14.817651 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260928 14:43:14.817771 16798 mvcc.cc:205] Tried to move back new op lower bound from 7334324612356612096 to 7334324612225830912. Current Snapshot: MvccSnapshot[applied={T|T < 7334324612356612096}]
I20260928 14:43:14.818087 16790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.817993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38209 (local address 127.9.33.66:39129)
0928 14:43:14.818028 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:14.818056 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260928 14:43:14.818361 16789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.818199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38209 (local address 127.9.33.65:42115)
0928 14:43:14.818308 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:14.818341 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:38209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260928 14:43:14.819267 16054 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:43:14.819367 16054 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:43:14.820660 16054 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 39a978772cff40a6882ec08ae1ae2f52: Sending DeleteTablet for 3 replicas of tablet 8e2e78fe96fa4663a27988d639f24cea
I20260928 14:43:14.821038 16353 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e2e78fe96fa4663a27988d639f24cea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:43:14 UTC) from {username='oryx', principal='oryx/127.9.33.125@KRBTEST.COM'} at 127.0.0.1:36390
I20260928 14:43:14.821230 16806 tablet_replica.cc:330] T 8e2e78fe96fa4663a27988d639f24cea P dc15cdcdf4b145588845d6ce64504136: stopping tablet replica
I20260928 14:43:14.821272 16626 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e2e78fe96fa4663a27988d639f24cea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:43:14 UTC) from {username='oryx', principal='oryx/127.9.33.125@KRBTEST.COM'} at 127.0.0.1:57492
I20260928 14:43:14.821317 16806 raft_consensus.cc:2229] T 8e2e78fe96fa4663a27988d639f24cea P dc15cdcdf4b145588845d6ce64504136 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:43:14.821365 16806 raft_consensus.cc:2258] T 8e2e78fe96fa4663a27988d639f24cea P dc15cdcdf4b145588845d6ce64504136 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:43:14.821341 16490 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e2e78fe96fa4663a27988d639f24cea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:43:14 UTC) from {username='oryx', principal='oryx/127.9.33.125@KRBTEST.COM'} at 127.0.0.1:40216
I20260928 14:43:14.821561 16807 tablet_replica.cc:330] T 8e2e78fe96fa4663a27988d639f24cea P 71899b6c516945508de4a4770b126e5f: stopping tablet replica
I20260928 14:43:14.821623 16807 raft_consensus.cc:2229] T 8e2e78fe96fa4663a27988d639f24cea P 71899b6c516945508de4a4770b126e5f [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:43:14.821646 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 16202
I20260928 14:43:14.821666 16806 ts_tablet_manager.cc:1905] T 8e2e78fe96fa4663a27988d639f24cea P dc15cdcdf4b145588845d6ce64504136: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:43:14.821708 16807 raft_consensus.cc:2258] T 8e2e78fe96fa4663a27988d639f24cea P 71899b6c516945508de4a4770b126e5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:43:14.822324 16807 ts_tablet_manager.cc:1905] T 8e2e78fe96fa4663a27988d639f24cea P 71899b6c516945508de4a4770b126e5f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:43:14.824079 16807 ts_tablet_manager.cc:1918] T 8e2e78fe96fa4663a27988d639f24cea P 71899b6c516945508de4a4770b126e5f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:43:14.824126 16807 log.cc:1196] T 8e2e78fe96fa4663a27988d639f24cea P 71899b6c516945508de4a4770b126e5f: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790606561545179-9349-0/minicluster-data/ts-2/wal/wals/8e2e78fe96fa4663a27988d639f24cea
I20260928 14:43:14.824298 16807 ts_tablet_manager.cc:1939] T 8e2e78fe96fa4663a27988d639f24cea P 71899b6c516945508de4a4770b126e5f: Deleting consensus metadata
I20260928 14:43:14.828148 16039 catalog_manager.cc:4841] TS 71899b6c516945508de4a4770b126e5f (127.9.33.67:34405): tablet 8e2e78fe96fa4663a27988d639f24cea (table test-table [id=a9b1c7a6af134133a7fa309293033d2c]) successfully deleted
I20260928 14:43:14.829208 16808 tablet_replica.cc:330] T 8e2e78fe96fa4663a27988d639f24cea P b5fdff97460e49f5acef24e0105aab26: stopping tablet replica
I20260928 14:43:14.829314 16808 raft_consensus.cc:2229] T 8e2e78fe96fa4663a27988d639f24cea P b5fdff97460e49f5acef24e0105aab26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:43:14.829380 16808 raft_consensus.cc:2258] T 8e2e78fe96fa4663a27988d639f24cea P b5fdff97460e49f5acef24e0105aab26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:43:14.829751 16808 ts_tablet_manager.cc:1905] T 8e2e78fe96fa4663a27988d639f24cea P b5fdff97460e49f5acef24e0105aab26: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260928 14:43:14.830624 16777 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:14.830386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:39055 (local address 127.0.0.1:36406)
0928 14:43:14.830523 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:14.830555 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.65:39055: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":111}
I20260928 14:43:14.830750 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 16427
W20260928 14:43:14.832666 16039 catalog_manager.cc:4584] TS dc15cdcdf4b145588845d6ce64504136 (127.9.33.65:39055): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8e2e78fe96fa4663a27988d639f24cea: Network error: Client connection negotiation failed: client connection to 127.9.33.65:39055: connect: Connection reset by peer (error 104)
I20260928 14:43:14.837874 16777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:14.837719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:37471 (local address 127.0.0.1:40224)
0928 14:43:14.837783 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:14.837820 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.66:37471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260928 14:43:14.837865 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 16563
I20260928 14:43:14.843194 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 15919
I20260928 14:43:14.847671 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 16108
2026-09-28T14:43:14Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (4767 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:43:14 dist-test-slave-pv0w krb5kdc[16811](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:43:14 dist-test-slave-pv0w krb5kdc[16811](info): set up 2 sockets
Sep 28 14:43:14 dist-test-slave-pv0w krb5kdc[16811](info): commencing operation
krb5kdc: starting...
W20260928 14:43:15.901027 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:43:15 dist-test-slave-pv0w krb5kdc[16811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:43:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:43:15Z Disabled control of system clock
WARNING: no policy specified for oryx/127.9.33.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.33.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.33.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.126.keytab.
Entry for principal oryx/127.9.33.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.126.keytab.
I20260928 14:43:16.031558 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:36113
--webserver_interface=127.9.33.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.126.keytab
--principal=oryx/127.9.33.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:33695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:36113
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:43:16.115087 16824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:16.115320 16824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:16.115343 16824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:16.116639 16824 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:43:16.116691 16824 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260928 14:43:16.116704 16824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:16.116719 16824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:16.116730 16824 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:43:16.116743 16824 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:43:16.118399 16824 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:33695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:36113
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.126.keytab
--principal=oryx/127.9.33.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:36113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:16.118702 16824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:16.118964 16824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 14:43:16.124500 16824 server_base.cc:1034] running on GCE node
W20260928 14:43:16.124634 16832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:16.124521 16830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:16.124514 16829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:16.124984 16824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:16.125327 16824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:16.126701 16824 hybrid_clock.cc:648] HybridClock initialized: now 1790606596126679 us; error 80 us; skew 500 ppm
Sep 28 14:43:16 dist-test-slave-pv0w krb5kdc[16811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606596, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:16.129753 16824 init.cc:377] Logged in from keytab as oryx/127.9.33.126@KRBTEST.COM (short username oryx)
I20260928 14:43:16.130406 16824 webserver.cc:460] Webserver started at http://127.9.33.126:39771/ using document root <none> and password file <none>
I20260928 14:43:16.130700 16824 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:16.130776 16824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:16.130937 16824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:16.132258 16824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "0c41cc0a857d4203a8750d8a6b304591"
format_stamp: "Formatted at 2026-09-28 14:43:16 on dist-test-slave-pv0w"
server_key: "5629abaa06a96be18c56ff20e35a8953"
server_key_iv: "dcb518054af80ffc7fba2c54383e27d8"
server_key_version: "encryptionkey@0"
I20260928 14:43:16.132656 16824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "0c41cc0a857d4203a8750d8a6b304591"
format_stamp: "Formatted at 2026-09-28 14:43:16 on dist-test-slave-pv0w"
server_key: "5629abaa06a96be18c56ff20e35a8953"
server_key_iv: "dcb518054af80ffc7fba2c54383e27d8"
server_key_version: "encryptionkey@0"
I20260928 14:43:16.134191 16824 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:43:16.135157 16839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:16.135438 16824 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:16.135561 16824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "0c41cc0a857d4203a8750d8a6b304591"
format_stamp: "Formatted at 2026-09-28 14:43:16 on dist-test-slave-pv0w"
server_key: "5629abaa06a96be18c56ff20e35a8953"
server_key_iv: "dcb518054af80ffc7fba2c54383e27d8"
server_key_version: "encryptionkey@0"
I20260928 14:43:16.135651 16824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:16.146198 16824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:16.148855 16824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:16.149026 16824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:16.153884 16824 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:36113
I20260928 14:43:16.154040 16901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:36113 every 8 connection(s)
I20260928 14:43:16.154348 16824 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:43:16.155113 16902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:16.156963 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 16824
I20260928 14:43:16.157089 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:43:16.157306 9349 external_mini_cluster.cc:1428] Setting key 7c0381802c8341cba67cd50ac970a379
I20260928 14:43:16.158020 16902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c41cc0a857d4203a8750d8a6b304591: Bootstrap starting.
I20260928 14:43:16.158682 16902 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c41cc0a857d4203a8750d8a6b304591: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:16.159072 16902 log.cc:824] T 00000000000000000000000000000000 P 0c41cc0a857d4203a8750d8a6b304591: Log is configured to *not* fsync() on all Append() calls
I20260928 14:43:16.159941 16902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c41cc0a857d4203a8750d8a6b304591: No bootstrap required, opened a new log
I20260928 14:43:16.162269 16902 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c41cc0a857d4203a8750d8a6b304591 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c41cc0a857d4203a8750d8a6b304591" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 36113 } }
I20260928 14:43:16.162514 16902 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c41cc0a857d4203a8750d8a6b304591 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:16.162557 16902 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c41cc0a857d4203a8750d8a6b304591 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c41cc0a857d4203a8750d8a6b304591, State: Initialized, Role: FOLLOWER
I20260928 14:43:16.162724 16902 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c41cc0a857d4203a8750d8a6b304591 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c41cc0a857d4203a8750d8a6b304591" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 36113 } }
I20260928 14:43:16.162808 16902 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c41cc0a857d4203a8750d8a6b304591 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:43:16.162832 16902 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c41cc0a857d4203a8750d8a6b304591 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:43:16.162901 16902 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c41cc0a857d4203a8750d8a6b304591 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:16.163776 16902 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c41cc0a857d4203a8750d8a6b304591 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c41cc0a857d4203a8750d8a6b304591" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 36113 } }
I20260928 14:43:16.163908 16902 leader_election.cc:302] T 00000000000000000000000000000000 P 0c41cc0a857d4203a8750d8a6b304591 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c41cc0a857d4203a8750d8a6b304591; no voters:
I20260928 14:43:16.164103 16902 leader_election.cc:288] T 00000000000000000000000000000000 P 0c41cc0a857d4203a8750d8a6b304591 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:43:16.164173 16907 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c41cc0a857d4203a8750d8a6b304591 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:43:16.164374 16907 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c41cc0a857d4203a8750d8a6b304591 [term 1 LEADER]: Becoming Leader. State: Replica: 0c41cc0a857d4203a8750d8a6b304591, State: Running, Role: LEADER
I20260928 14:43:16.164407 16902 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c41cc0a857d4203a8750d8a6b304591 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:43:16.164491 16907 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c41cc0a857d4203a8750d8a6b304591 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c41cc0a857d4203a8750d8a6b304591" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 36113 } }
Sep 28 14:43:16 dist-test-slave-pv0w krb5kdc[16811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM
I20260928 14:43:16.164822 16907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c41cc0a857d4203a8750d8a6b304591 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c41cc0a857d4203a8750d8a6b304591. Latest consensus state: current_term: 1 leader_uuid: "0c41cc0a857d4203a8750d8a6b304591" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c41cc0a857d4203a8750d8a6b304591" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 36113 } } }
I20260928 14:43:16.164902 16907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c41cc0a857d4203a8750d8a6b304591 [sys.catalog]: This master's current role is: LEADER
I20260928 14:43:16.165048 16909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c41cc0a857d4203a8750d8a6b304591 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c41cc0a857d4203a8750d8a6b304591" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c41cc0a857d4203a8750d8a6b304591" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 36113 } } }
I20260928 14:43:16.165112 16909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c41cc0a857d4203a8750d8a6b304591 [sys.catalog]: This master's current role is: LEADER
I20260928 14:43:16.165230 16913 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:43:16.165978 16913 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:43:16.167676 16913 catalog_manager.cc:1347] Generated new cluster ID: 243b4b15931047dca25b9693cc72cea0
I20260928 14:43:16.167747 16913 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:43:16.169293 16905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:16.157884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46682 (local address 127.9.33.126:36113)
0928 14:43:16.158105 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:16.158114 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:16.158167 (+ 53us) server_negotiation.cc:407] Connection header received
0928 14:43:16.158662 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:16.158670 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:16.158719 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:16.158901 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:16.159695 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.160706 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.161832 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.162274 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.162364 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:16.165466 (+ 3102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:16.165470 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:16.165471 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:16.165485 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:16.167536 (+ 2051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:16.167982 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:16.167986 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:16.167986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:16.168096 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:16.168327 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:16.168328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:16.168328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:16.168620 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:16.168753 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:16.168918 (+ 165us) server_negotiation.cc:299] Negotiation successful
0928 14:43:16.169061 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":108,"threads_started":1}
I20260928 14:43:16.176200 16913 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:43:16.176945 16913 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:43:16.192925 16913 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c41cc0a857d4203a8750d8a6b304591: Generated new TSK 0
I20260928 14:43:16.193188 16913 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.9.33.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.33.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.33.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.65.keytab.
Entry for principal oryx/127.9.33.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.65.keytab.
I20260928 14:43:16.231422 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.65:0
--local_ip_for_outbound_sockets=127.9.33.65
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:36113
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.65.keytab
--principal=oryx/127.9.33.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:33695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:43:16.317813 16928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:16.318039 16928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:16.318068 16928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:16.319334 16928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:16.319381 16928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:16.319404 16928 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 14:43:16.319423 16928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.65
I20260928 14:43:16.321139 16928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:33695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.65.keytab
--principal=oryx/127.9.33.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.33.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.33.126:36113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.65
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:16.321415 16928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:16.321683 16928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:16.327333 16936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:16.327423 16933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:16.327323 16934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:16.327602 16928 server_base.cc:1034] running on GCE node
I20260928 14:43:16.327896 16928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:16.328142 16928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:16.329347 16928 hybrid_clock.cc:648] HybridClock initialized: now 1790606596329288 us; error 75 us; skew 500 ppm
Sep 28 14:43:16 dist-test-slave-pv0w krb5kdc[16811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606596, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:16.332422 16928 init.cc:377] Logged in from keytab as oryx/127.9.33.65@KRBTEST.COM (short username oryx)
I20260928 14:43:16.332970 16928 webserver.cc:460] Webserver started at http://127.9.33.65:39709/ using document root <none> and password file <none>
I20260928 14:43:16.333348 16928 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:16.333426 16928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:16.333616 16928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:16.334836 16928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-0/data/instance:
uuid: "fb063f3bf2c24ca29f5480118eaa718b"
format_stamp: "Formatted at 2026-09-28 14:43:16 on dist-test-slave-pv0w"
server_key: "ccbdaf4bdc362a25497f0c0b4832a839"
server_key_iv: "178091b79ffc4c232eaed775376669e4"
server_key_version: "encryptionkey@0"
I20260928 14:43:16.335414 16928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance:
uuid: "fb063f3bf2c24ca29f5480118eaa718b"
format_stamp: "Formatted at 2026-09-28 14:43:16 on dist-test-slave-pv0w"
server_key: "ccbdaf4bdc362a25497f0c0b4832a839"
server_key_iv: "178091b79ffc4c232eaed775376669e4"
server_key_version: "encryptionkey@0"
I20260928 14:43:16.337215 16928 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260928 14:43:16.338194 16943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:16.338449 16928 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:43:16.338584 16928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-0/wal
uuid: "fb063f3bf2c24ca29f5480118eaa718b"
format_stamp: "Formatted at 2026-09-28 14:43:16 on dist-test-slave-pv0w"
server_key: "ccbdaf4bdc362a25497f0c0b4832a839"
server_key_iv: "178091b79ffc4c232eaed775376669e4"
server_key_version: "encryptionkey@0"
I20260928 14:43:16.338680 16928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:16.349618 16928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:16.352104 16928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:16.352295 16928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:16.353013 16928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:43:16.353083 16928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:16.353132 16928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:43:16.353160 16928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 14:43:16 dist-test-slave-pv0w krb5kdc[16811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606596, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM
I20260928 14:43:16.360510 16928 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.65:33699
I20260928 14:43:16.360961 16928 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
I20260928 14:43:16.360965 17058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.65:33699 every 8 connection(s)
I20260928 14:43:16.364138 16905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:16.353484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:36519 (local address 127.9.33.126:36113)
0928 14:43:16.353560 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:16.353563 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:16.353808 (+ 245us) server_negotiation.cc:407] Connection header received
0928 14:43:16.354348 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:16.354349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:16.354364 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:16.354443 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:16.355466 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.356386 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.357605 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.357972 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.358061 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:16.360923 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:16.360925 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:16.360925 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:16.360934 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:16.362570 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:16.363032 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:16.363033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:16.363034 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:16.363105 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:16.363368 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:16.363369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:16.363370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:16.363501 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:16.363599 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:16.363947 (+ 348us) server_negotiation.cc:299] Negotiation successful
0928 14:43:16.364043 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260928 14:43:16.364377 16952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:16.353484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:36113 (local address 127.9.33.65:36519)
0928 14:43:16.353664 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:16.353709 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:16.354239 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:16.354560 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:16.354564 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:16.355015 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:16.355335 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:16.355344 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.356499 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.356500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:16.357533 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:16.357535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.358081 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.358083 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:16.358225 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:16.359224 (+ 999us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:16.359233 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:16.360814 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:16.362684 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:16.362691 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:16.362693 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:16.362937 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:16.363172 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:16.363173 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:16.363173 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:16.363292 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:16.363589 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:16.363596 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:16.363799 (+ 203us) client_negotiation.cc:769] Sending connection context
0928 14:43:16.363928 (+ 129us) client_negotiation.cc:240] Negotiation successful
0928 14:43:16.364106 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":88,"threads_started":1}
I20260928 14:43:16.364909 17059 heartbeater.cc:347] Connected to a master server at 127.9.33.126:36113
I20260928 14:43:16.365024 17059 heartbeater.cc:464] Registering TS with master...
I20260928 14:43:16.365273 17059 heartbeater.cc:511] Master 127.9.33.126:36113 requested a full tablet report, sending...
I20260928 14:43:16.365841 16856 ts_manager.cc:194] Registered new tserver with Master: fb063f3bf2c24ca29f5480118eaa718b (127.9.33.65:33699)
I20260928 14:43:16.366734 16856 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.33.65@KRBTEST.COM'} at 127.9.33.65:36519
I20260928 14:43:16.367520 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 16928
I20260928 14:43:16.367647 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance
I20260928 14:43:16.367889 9349 external_mini_cluster.cc:1428] Setting key e6978561f61c000f6355262162188213
I20260928 14:43:16.373039 17065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:16.366020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:36113 (local address 127.9.33.65:45581)
0928 14:43:16.366140 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:16.366150 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:16.366208 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:16.366351 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:16.366352 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:16.366520 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:16.366574 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:16.366575 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.367432 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.367433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:16.368306 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:16.368309 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.368742 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.368743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:16.368831 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:16.369259 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:16.369265 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:16.369710 (+ 445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:16.371711 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:16.371712 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:16.371712 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:16.372001 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:16.372276 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:16.372277 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:16.372278 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:16.372350 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:16.372693 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:16.372695 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:16.372775 (+ 80us) client_negotiation.cc:769] Sending connection context
0928 14:43:16.372860 (+ 85us) client_negotiation.cc:240] Negotiation successful
0928 14:43:16.372962 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":70,"threads_started":1}
I20260928 14:43:16.373117 16905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:16.366080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:45581 (local address 127.9.33.126:36113)
0928 14:43:16.366121 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:16.366123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:16.366194 (+ 71us) server_negotiation.cc:407] Connection header received
0928 14:43:16.366269 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:16.366269 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:16.366279 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:16.366345 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:16.366645 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.367289 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.368391 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.368660 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.368718 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:16.369827 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:16.369828 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:16.369828 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:16.369835 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:16.371590 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:16.372083 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:16.372084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:16.372085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:16.372154 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:16.372428 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:16.372429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:16.372430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:16.372597 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:16.372676 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:16.372910 (+ 234us) server_negotiation.cc:299] Negotiation successful
0928 14:43:16.373009 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
WARNING: no policy specified for oryx/127.9.33.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.33.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.33.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.66.keytab.
Entry for principal oryx/127.9.33.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.66.keytab.
I20260928 14:43:16.395293 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.66:0
--local_ip_for_outbound_sockets=127.9.33.66
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:36113
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.66.keytab
--principal=oryx/127.9.33.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:33695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:43:16.480124 17068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:16.480352 17068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:16.480378 17068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:16.481527 17068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:16.481570 17068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:16.481590 17068 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 14:43:16.481609 17068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.66
I20260928 14:43:16.483112 17068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:33695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.66.keytab
--principal=oryx/127.9.33.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.33.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.33.126:36113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.66
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:16.483309 17068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:16.483492 17068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:16.488791 17074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:16.488791 17073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:16.489008 17068 server_base.cc:1034] running on GCE node
W20260928 14:43:16.488884 17076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:16.489349 17068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:16.489650 17068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:16.490831 17068 hybrid_clock.cc:648] HybridClock initialized: now 1790606596490789 us; error 59 us; skew 500 ppm
Sep 28 14:43:16 dist-test-slave-pv0w krb5kdc[16811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606596, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:16.493618 17068 init.cc:377] Logged in from keytab as oryx/127.9.33.66@KRBTEST.COM (short username oryx)
I20260928 14:43:16.494170 17068 webserver.cc:460] Webserver started at http://127.9.33.66:39395/ using document root <none> and password file <none>
I20260928 14:43:16.494412 17068 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:16.494490 17068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:16.494650 17068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:16.495869 17068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-1/data/instance:
uuid: "7f8b53fbbb804e189ce714679eb3e35c"
format_stamp: "Formatted at 2026-09-28 14:43:16 on dist-test-slave-pv0w"
server_key: "e42d5f8cdb0a902c8359f614fbe773fa"
server_key_iv: "cf64893a144ac87592de056d7dc6f0f0"
server_key_version: "encryptionkey@0"
I20260928 14:43:16.496243 17068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance:
uuid: "7f8b53fbbb804e189ce714679eb3e35c"
format_stamp: "Formatted at 2026-09-28 14:43:16 on dist-test-slave-pv0w"
server_key: "e42d5f8cdb0a902c8359f614fbe773fa"
server_key_iv: "cf64893a144ac87592de056d7dc6f0f0"
server_key_version: "encryptionkey@0"
I20260928 14:43:16.497682 17068 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260928 14:43:16.498462 17083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:16.498652 17068 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:43:16.498761 17068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-1/wal
uuid: "7f8b53fbbb804e189ce714679eb3e35c"
format_stamp: "Formatted at 2026-09-28 14:43:16 on dist-test-slave-pv0w"
server_key: "e42d5f8cdb0a902c8359f614fbe773fa"
server_key_iv: "cf64893a144ac87592de056d7dc6f0f0"
server_key_version: "encryptionkey@0"
I20260928 14:43:16.498840 17068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:16.519364 17068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:16.521730 17068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:16.521906 17068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:16.522580 17068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:43:16.522631 17068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:16.522668 17068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:43:16.522692 17068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 28 14:43:16 dist-test-slave-pv0w krb5kdc[16811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606596, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM
I20260928 14:43:16.530845 17068 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.66:44523
I20260928 14:43:16.531101 17198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.66:44523 every 8 connection(s)
I20260928 14:43:16.531494 17068 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
I20260928 14:43:16.534255 16905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:16.523100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:52227 (local address 127.9.33.126:36113)
0928 14:43:16.523174 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:16.523176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:16.523431 (+ 255us) server_negotiation.cc:407] Connection header received
0928 14:43:16.524014 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:16.524016 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:16.524032 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:16.524116 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:16.524807 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.525671 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.526866 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.527145 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.527222 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:16.530620 (+ 3398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:16.530622 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:16.530622 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:16.530631 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:16.532676 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:16.533188 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:16.533190 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:16.533190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:16.533262 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:16.533531 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:16.533532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:16.533532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:16.533671 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:16.533746 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:16.534052 (+ 306us) server_negotiation.cc:299] Negotiation successful
0928 14:43:16.534147 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260928 14:43:16.534518 17092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:16.523083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:36113 (local address 127.9.33.66:52227)
0928 14:43:16.523258 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:16.523326 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:16.523902 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:16.524219 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:16.524223 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:16.524489 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:16.524710 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:16.524715 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.525773 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.525775 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:16.526737 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:16.526740 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.527247 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.527248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:16.527435 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:16.528201 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:16.528216 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:16.530486 (+ 2270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:16.532818 (+ 2332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:16.532826 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:16.532828 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:16.533096 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:16.533331 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:16.533332 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:16.533333 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:16.533452 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:16.533749 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:16.533756 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:16.533913 (+ 157us) client_negotiation.cc:769] Sending connection context
0928 14:43:16.534030 (+ 117us) client_negotiation.cc:240] Negotiation successful
0928 14:43:16.534232 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":99,"threads_started":1}
I20260928 14:43:16.535079 17199 heartbeater.cc:347] Connected to a master server at 127.9.33.126:36113
I20260928 14:43:16.535161 17199 heartbeater.cc:464] Registering TS with master...
I20260928 14:43:16.535369 17199 heartbeater.cc:511] Master 127.9.33.126:36113 requested a full tablet report, sending...
I20260928 14:43:16.535862 16855 ts_manager.cc:194] Registered new tserver with Master: 7f8b53fbbb804e189ce714679eb3e35c (127.9.33.66:44523)
I20260928 14:43:16.536451 16855 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.33.66@KRBTEST.COM'} at 127.9.33.66:52227
I20260928 14:43:16.540980 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 17068
I20260928 14:43:16.541124 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance
I20260928 14:43:16.541385 9349 external_mini_cluster.cc:1428] Setting key ce0775a6f120ba06a973dc3ed1cd59d0
I20260928 14:43:16.543610 17205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:16.536246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:36113 (local address 127.9.33.66:55541)
0928 14:43:16.536388 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:16.536400 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:16.536465 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:16.536621 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:16.536622 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:16.536865 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:16.536928 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:16.536930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.537757 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.537758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:16.538346 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:16.538347 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.538827 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.538828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:16.538935 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:16.539533 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:16.539538 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:16.539999 (+ 461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:16.542015 (+ 2016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:16.542016 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:16.542016 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:16.542212 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:16.542467 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:16.542468 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:16.542468 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:16.542537 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:16.543257 (+ 720us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:16.543258 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:16.543322 (+ 64us) client_negotiation.cc:769] Sending connection context
0928 14:43:16.543420 (+ 98us) client_negotiation.cc:240] Negotiation successful
0928 14:43:16.543532 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":86,"threads_started":1}
I20260928 14:43:16.543780 16905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:16.536343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:55541 (local address 127.9.33.126:36113)
0928 14:43:16.536390 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:16.536392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:16.536450 (+ 58us) server_negotiation.cc:407] Connection header received
0928 14:43:16.536524 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:16.536525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:16.536533 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:16.536614 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:16.537043 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.537640 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.538486 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.538742 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.538812 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:16.540099 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:16.540101 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:16.540101 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:16.540108 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:16.541912 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:16.542314 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:16.542316 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:16.542316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:16.542405 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:16.542607 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:16.542607 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:16.542608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:16.543138 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:16.543241 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:16.543500 (+ 259us) server_negotiation.cc:299] Negotiation successful
0928 14:43:16.543605 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
WARNING: no policy specified for oryx/127.9.33.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.33.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.33.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.67.keytab.
Entry for principal oryx/127.9.33.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.67.keytab.
I20260928 14:43:16.571884 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.67:0
--local_ip_for_outbound_sockets=127.9.33.67
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:36113
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.67.keytab
--principal=oryx/127.9.33.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:33695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:43:16.656075 17208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:16.656353 17208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:16.656399 17208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:16.657627 17208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:16.657701 17208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:16.657737 17208 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260928 14:43:16.657780 17208 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.67
I20260928 14:43:16.659458 17208 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:33695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.67.keytab
--principal=oryx/127.9.33.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.33.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.33.126:36113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.67
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:16.659847 17208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:16.660099 17208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:16.665587 17213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:16.665593 17214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:16.665593 17216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:16.665812 17208 server_base.cc:1034] running on GCE node
I20260928 14:43:16.666154 17208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:16.666424 17208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:16.667609 17208 hybrid_clock.cc:648] HybridClock initialized: now 1790606596667590 us; error 30 us; skew 500 ppm
Sep 28 14:43:16 dist-test-slave-pv0w krb5kdc[16811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606596, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:16.670447 17208 init.cc:377] Logged in from keytab as oryx/127.9.33.67@KRBTEST.COM (short username oryx)
I20260928 14:43:16.671013 17208 webserver.cc:460] Webserver started at http://127.9.33.67:40171/ using document root <none> and password file <none>
I20260928 14:43:16.671276 17208 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:16.671348 17208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:16.671509 17208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:16.672711 17208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-2/data/instance:
uuid: "1671965ee3334946868d67415747aba5"
format_stamp: "Formatted at 2026-09-28 14:43:16 on dist-test-slave-pv0w"
server_key: "737b1436df396439c9a4e1e4e729bfab"
server_key_iv: "29b3f0c5b7f20240db748924a75f052a"
server_key_version: "encryptionkey@0"
I20260928 14:43:16.673146 17208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance:
uuid: "1671965ee3334946868d67415747aba5"
format_stamp: "Formatted at 2026-09-28 14:43:16 on dist-test-slave-pv0w"
server_key: "737b1436df396439c9a4e1e4e729bfab"
server_key_iv: "29b3f0c5b7f20240db748924a75f052a"
server_key_version: "encryptionkey@0"
I20260928 14:43:16.674528 17208 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:16.675324 17223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:16.675493 17208 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:43:16.675611 17208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-2/wal
uuid: "1671965ee3334946868d67415747aba5"
format_stamp: "Formatted at 2026-09-28 14:43:16 on dist-test-slave-pv0w"
server_key: "737b1436df396439c9a4e1e4e729bfab"
server_key_iv: "29b3f0c5b7f20240db748924a75f052a"
server_key_version: "encryptionkey@0"
I20260928 14:43:16.675691 17208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:16.696336 17208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:16.698491 17208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:16.698679 17208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:16.699218 17208 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:43:16.699285 17208 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:16.699331 17208 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:43:16.699362 17208 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:16.706943 17208 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.67:35545
I20260928 14:43:16.707027 17338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.67:35545 every 8 connection(s)
Sep 28 14:43:16 dist-test-slave-pv0w krb5kdc[16811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606596, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM
I20260928 14:43:16.707377 17208 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
I20260928 14:43:16.707674 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 17208
I20260928 14:43:16.707818 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance
I20260928 14:43:16.708030 9349 external_mini_cluster.cc:1428] Setting key 59513e1cf5134e13e38ecbcecd039581
I20260928 14:43:16.710836 16905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:16.699816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:48865 (local address 127.9.33.126:36113)
0928 14:43:16.699875 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:16.699878 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:16.700109 (+ 231us) server_negotiation.cc:407] Connection header received
0928 14:43:16.700625 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:16.700627 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:16.700643 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:16.700726 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:16.701708 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.702595 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.704427 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.704767 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.704831 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:16.707890 (+ 3059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:16.707892 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:16.707892 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:16.707901 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:16.709454 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:16.709885 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:16.709886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:16.709886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:16.709938 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:16.710214 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:16.710216 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:16.710216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:16.710350 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:16.710449 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:16.710692 (+ 243us) server_negotiation.cc:299] Negotiation successful
0928 14:43:16.710761 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260928 14:43:16.711004 17232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:16.699819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:36113 (local address 127.9.33.67:48865)
0928 14:43:16.699971 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:16.700014 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:16.700525 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:16.700890 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:16.700894 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:16.701291 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:16.701593 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:16.701600 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.702722 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.702724 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:16.704294 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:16.704298 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.704907 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.704908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:16.705088 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:16.705754 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:16.705765 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:16.707775 (+ 2010us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:16.709547 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:16.709554 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:16.709556 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:16.709804 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:16.709997 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:16.709998 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:16.709998 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:16.710115 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:16.710444 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:16.710449 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:16.710570 (+ 121us) client_negotiation.cc:769] Sending connection context
0928 14:43:16.710646 (+ 76us) client_negotiation.cc:240] Negotiation successful
0928 14:43:16.710776 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":82,"threads_started":1}
I20260928 14:43:16.711513 17339 heartbeater.cc:347] Connected to a master server at 127.9.33.126:36113
I20260928 14:43:16.711601 17339 heartbeater.cc:464] Registering TS with master...
I20260928 14:43:16.711840 17339 heartbeater.cc:511] Master 127.9.33.126:36113 requested a full tablet report, sending...
I20260928 14:43:16.712225 16855 ts_manager.cc:194] Registered new tserver with Master: 1671965ee3334946868d67415747aba5 (127.9.33.67:35545)
I20260928 14:43:16.712742 16855 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.33.67@KRBTEST.COM'} at 127.9.33.67:48865
I20260928 14:43:16.718312 16905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:16.712883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:47953 (local address 127.9.33.126:36113)
0928 14:43:16.712927 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:16.712930 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:16.712968 (+ 38us) server_negotiation.cc:407] Connection header received
0928 14:43:16.713040 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:16.713041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:16.713051 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:16.713107 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:16.713450 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.713942 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.714659 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.714890 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.714977 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:16.715898 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:16.715899 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:16.715900 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:16.715907 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:16.717145 (+ 1238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:16.717513 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:16.717515 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:16.717515 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:16.717576 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:16.717773 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:16.717774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:16.717774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:16.717898 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:16.717971 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:16.718139 (+ 168us) server_negotiation.cc:299] Negotiation successful
0928 14:43:16.718216 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 14:43:16.718317 17345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:16.712710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:36113 (local address 127.9.33.67:47953)
0928 14:43:16.712911 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:16.712923 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:16.712983 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:16.713112 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:16.713113 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:16.713301 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:16.713369 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:16.713370 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.714037 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.714039 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:16.714575 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:16.714577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.714979 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.714980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:16.715057 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:16.715457 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:16.715461 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:16.715801 (+ 340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:16.717244 (+ 1443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:16.717245 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:16.717246 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:16.717425 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:16.717642 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:16.717643 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:16.717643 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:16.717708 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:16.717977 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:16.717978 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:16.718024 (+ 46us) client_negotiation.cc:769] Sending connection context
0928 14:43:16.718109 (+ 85us) client_negotiation.cc:240] Negotiation successful
0928 14:43:16.718217 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":131,"threads_started":1}
I20260928 14:43:16.719918 9349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 28 14:43:16 dist-test-slave-pv0w krb5kdc[16811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606596, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 28 14:43:16 dist-test-slave-pv0w krb5kdc[16811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:16 dist-test-slave-pv0w krb5kdc[16811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
W20260928 14:43:16.741671 16905 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:16.735789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46694 (local address 127.9.33.126:36113)
0928 14:43:16.735864 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:16.735867 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:16.735876 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:43:16.735910 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:16.735911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:16.735921 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:16.735985 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:16.736648 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.737165 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.737971 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.738168 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.738247 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:16.741562 (+ 3315us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46694: BlockingRecv error: recv got EOF from 127.0.0.1:46694 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Sep 28 14:43:16 dist-test-slave-pv0w krb5kdc[16811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606596, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM
I20260928 14:43:16.750082 16905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:16.742518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46696 (local address 127.9.33.126:36113)
0928 14:43:16.742561 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:16.742563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:16.742734 (+ 171us) server_negotiation.cc:407] Connection header received
0928 14:43:16.742816 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:16.742817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:16.742827 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:16.742927 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:16.743441 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.744022 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.744737 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.744946 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.745029 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:16.747367 (+ 2338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:16.747369 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:16.747369 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:16.747378 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:16.748870 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:16.749252 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:16.749253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:16.749253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:16.749317 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:16.749535 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:16.749536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:16.749536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:16.749658 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:16.749764 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:16.749883 (+ 119us) server_negotiation.cc:299] Negotiation successful
0928 14:43:16.749970 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260928 14:43:16.752115 16855 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46696:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:43:16.753906 16855 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:43:16.761171 17361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:16.756553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:33699 (local address 127.0.0.1:54694)
0928 14:43:16.757086 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:16.757095 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:16.757174 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:16.757472 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:16.757476 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:16.757479 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:43:16.757584 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:16.757587 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.758550 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.758551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:16.759884 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:16.759887 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.760828 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.760829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:16.760919 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:16.760936 (+ 17us) client_negotiation.cc:769] Sending connection context
0928 14:43:16.761009 (+ 73us) client_negotiation.cc:240] Negotiation successful
0928 14:43:16.761078 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"mutex_wait_us":4,"thread_start_us":90,"threads_started":1}
I20260928 14:43:16.761298 17363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:16.756546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:35545 (local address 127.0.0.1:36406)
0928 14:43:16.757040 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:16.757086 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:16.757174 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:16.757472 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:16.757476 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:16.757481 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:43:16.757620 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:16.757621 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.759013 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.759014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:16.760097 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:16.760099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.761001 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.761002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:16.761073 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:16.761078 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:43:16.761135 (+ 57us) client_negotiation.cc:240] Negotiation successful
0928 14:43:16.761182 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"mutex_wait_us":56,"thread_start_us":73,"threads_started":1}
I20260928 14:43:16.761291 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:16.756637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36406 (local address 127.9.33.67:35545)
0928 14:43:16.756788 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:16.756793 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:16.757215 (+ 422us) server_negotiation.cc:407] Connection header received
0928 14:43:16.757274 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:16.757276 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:16.757292 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:16.757384 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:43:16.757700 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.758914 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.760189 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.760932 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.760999 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:16.761066 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:16.761151 (+ 85us) server_negotiation.cc:299] Negotiation successful
0928 14:43:16.761186 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":84,"threads_started":1}
I20260928 14:43:16.761569 17366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:16.756815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54694 (local address 127.9.33.65:33699)
0928 14:43:16.757047 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:16.757051 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:16.757184 (+ 133us) server_negotiation.cc:407] Connection header received
0928 14:43:16.757269 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:16.757272 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:16.757290 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:16.757384 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:43:16.757677 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.758455 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.759975 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.760713 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.760826 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:16.760913 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:16.761428 (+ 515us) server_negotiation.cc:299] Negotiation successful
0928 14:43:16.761481 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":123,"threads_started":1}
I20260928 14:43:16.762230 16993 tablet_service.cc:1467] Processing CreateTablet for tablet 61d0deb8b2f14981995b7c786d9e48cf (DEFAULT_TABLE table=test-table [id=cf2ee29f1f9545dc89d131fe8b910bca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:43:16.762360 17273 tablet_service.cc:1467] Processing CreateTablet for tablet 61d0deb8b2f14981995b7c786d9e48cf (DEFAULT_TABLE table=test-table [id=cf2ee29f1f9545dc89d131fe8b910bca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:43:16.762651 17362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:16.756553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:44523 (local address 127.0.0.1:39600)
0928 14:43:16.757339 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:16.757348 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:16.757393 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:16.757685 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:16.757686 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:16.757689 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:43:16.757751 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:16.757752 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.759996 (+ 2244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.759996 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:16.760746 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:16.760748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.762341 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.762343 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:16.762433 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:16.762438 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:43:16.762518 (+ 80us) client_negotiation.cc:240] Negotiation successful
0928 14:43:16.762572 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":769,"mutex_wait_us":3,"thread_start_us":97,"threads_started":1}
I20260928 14:43:16.762720 16993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61d0deb8b2f14981995b7c786d9e48cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:16.762787 17273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61d0deb8b2f14981995b7c786d9e48cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:16.762967 17365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:16.756815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39600 (local address 127.9.33.66:44523)
0928 14:43:16.757129 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:16.757132 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:16.757451 (+ 319us) server_negotiation.cc:407] Connection header received
0928 14:43:16.757496 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:16.757499 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:16.757516 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:16.757583 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:43:16.757858 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.758926 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.760879 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.762210 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.762307 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:16.762389 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:16.762659 (+ 270us) server_negotiation.cc:299] Negotiation successful
0928 14:43:16.762858 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":120,"threads_started":1}
I20260928 14:43:16.765535 17133 tablet_service.cc:1467] Processing CreateTablet for tablet 61d0deb8b2f14981995b7c786d9e48cf (DEFAULT_TABLE table=test-table [id=cf2ee29f1f9545dc89d131fe8b910bca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:43:16.765966 17133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61d0deb8b2f14981995b7c786d9e48cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:16.766472 17367 tablet_bootstrap.cc:492] T 61d0deb8b2f14981995b7c786d9e48cf P 1671965ee3334946868d67415747aba5: Bootstrap starting.
I20260928 14:43:16.767133 17367 tablet_bootstrap.cc:654] T 61d0deb8b2f14981995b7c786d9e48cf P 1671965ee3334946868d67415747aba5: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:16.767458 17367 log.cc:824] T 61d0deb8b2f14981995b7c786d9e48cf P 1671965ee3334946868d67415747aba5: Log is configured to *not* fsync() on all Append() calls
I20260928 14:43:16.767980 17368 tablet_bootstrap.cc:492] T 61d0deb8b2f14981995b7c786d9e48cf P fb063f3bf2c24ca29f5480118eaa718b: Bootstrap starting.
I20260928 14:43:16.768337 17367 tablet_bootstrap.cc:492] T 61d0deb8b2f14981995b7c786d9e48cf P 1671965ee3334946868d67415747aba5: No bootstrap required, opened a new log
I20260928 14:43:16.768422 17367 ts_tablet_manager.cc:1397] T 61d0deb8b2f14981995b7c786d9e48cf P 1671965ee3334946868d67415747aba5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:43:16.768663 17368 tablet_bootstrap.cc:654] T 61d0deb8b2f14981995b7c786d9e48cf P fb063f3bf2c24ca29f5480118eaa718b: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:16.768999 17368 log.cc:824] T 61d0deb8b2f14981995b7c786d9e48cf P fb063f3bf2c24ca29f5480118eaa718b: Log is configured to *not* fsync() on all Append() calls
I20260928 14:43:16.769842 17368 tablet_bootstrap.cc:492] T 61d0deb8b2f14981995b7c786d9e48cf P fb063f3bf2c24ca29f5480118eaa718b: No bootstrap required, opened a new log
I20260928 14:43:16.769937 17368 ts_tablet_manager.cc:1397] T 61d0deb8b2f14981995b7c786d9e48cf P fb063f3bf2c24ca29f5480118eaa718b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:43:16.769877 17371 tablet_bootstrap.cc:492] T 61d0deb8b2f14981995b7c786d9e48cf P 7f8b53fbbb804e189ce714679eb3e35c: Bootstrap starting.
I20260928 14:43:16.770529 17371 tablet_bootstrap.cc:654] T 61d0deb8b2f14981995b7c786d9e48cf P 7f8b53fbbb804e189ce714679eb3e35c: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:16.770632 17367 raft_consensus.cc:348] T 61d0deb8b2f14981995b7c786d9e48cf P 1671965ee3334946868d67415747aba5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } }
I20260928 14:43:16.770830 17367 raft_consensus.cc:374] T 61d0deb8b2f14981995b7c786d9e48cf P 1671965ee3334946868d67415747aba5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:16.770885 17367 raft_consensus.cc:729] T 61d0deb8b2f14981995b7c786d9e48cf P 1671965ee3334946868d67415747aba5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1671965ee3334946868d67415747aba5, State: Initialized, Role: FOLLOWER
I20260928 14:43:16.770877 17371 log.cc:824] T 61d0deb8b2f14981995b7c786d9e48cf P 7f8b53fbbb804e189ce714679eb3e35c: Log is configured to *not* fsync() on all Append() calls
I20260928 14:43:16.771040 17367 consensus_queue.cc:260] T 61d0deb8b2f14981995b7c786d9e48cf P 1671965ee3334946868d67415747aba5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } }
I20260928 14:43:16.771359 17367 ts_tablet_manager.cc:1428] T 61d0deb8b2f14981995b7c786d9e48cf P 1671965ee3334946868d67415747aba5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:43:16.771431 17339 heartbeater.cc:503] Master 127.9.33.126:36113 was elected leader, sending a full tablet report...
I20260928 14:43:16.771910 17371 tablet_bootstrap.cc:492] T 61d0deb8b2f14981995b7c786d9e48cf P 7f8b53fbbb804e189ce714679eb3e35c: No bootstrap required, opened a new log
I20260928 14:43:16.772006 17371 ts_tablet_manager.cc:1397] T 61d0deb8b2f14981995b7c786d9e48cf P 7f8b53fbbb804e189ce714679eb3e35c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:43:16.772125 17368 raft_consensus.cc:348] T 61d0deb8b2f14981995b7c786d9e48cf P fb063f3bf2c24ca29f5480118eaa718b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } }
I20260928 14:43:16.772274 17368 raft_consensus.cc:374] T 61d0deb8b2f14981995b7c786d9e48cf P fb063f3bf2c24ca29f5480118eaa718b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:16.772296 17368 raft_consensus.cc:729] T 61d0deb8b2f14981995b7c786d9e48cf P fb063f3bf2c24ca29f5480118eaa718b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb063f3bf2c24ca29f5480118eaa718b, State: Initialized, Role: FOLLOWER
I20260928 14:43:16.772425 17368 consensus_queue.cc:260] T 61d0deb8b2f14981995b7c786d9e48cf P fb063f3bf2c24ca29f5480118eaa718b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } }
I20260928 14:43:16.772702 17368 ts_tablet_manager.cc:1428] T 61d0deb8b2f14981995b7c786d9e48cf P fb063f3bf2c24ca29f5480118eaa718b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260928 14:43:16.772830 17059 heartbeater.cc:503] Master 127.9.33.126:36113 was elected leader, sending a full tablet report...
I20260928 14:43:16.774205 17371 raft_consensus.cc:348] T 61d0deb8b2f14981995b7c786d9e48cf P 7f8b53fbbb804e189ce714679eb3e35c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } }
I20260928 14:43:16.774325 17371 raft_consensus.cc:374] T 61d0deb8b2f14981995b7c786d9e48cf P 7f8b53fbbb804e189ce714679eb3e35c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:16.774346 17371 raft_consensus.cc:729] T 61d0deb8b2f14981995b7c786d9e48cf P 7f8b53fbbb804e189ce714679eb3e35c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f8b53fbbb804e189ce714679eb3e35c, State: Initialized, Role: FOLLOWER
I20260928 14:43:16.774433 17371 consensus_queue.cc:260] T 61d0deb8b2f14981995b7c786d9e48cf P 7f8b53fbbb804e189ce714679eb3e35c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } }
I20260928 14:43:16.774632 17371 ts_tablet_manager.cc:1428] T 61d0deb8b2f14981995b7c786d9e48cf P 7f8b53fbbb804e189ce714679eb3e35c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:43:16.774725 17199 heartbeater.cc:503] Master 127.9.33.126:36113 was elected leader, sending a full tablet report...
W20260928 14:43:16.782171 17200 tablet.cc:2369] T 61d0deb8b2f14981995b7c786d9e48cf P 7f8b53fbbb804e189ce714679eb3e35c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:43:16.861877 17060 tablet.cc:2369] T 61d0deb8b2f14981995b7c786d9e48cf P fb063f3bf2c24ca29f5480118eaa718b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:43:16.879616 17373 raft_consensus.cc:482] T 61d0deb8b2f14981995b7c786d9e48cf P 1671965ee3334946868d67415747aba5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:43:16.879801 17373 raft_consensus.cc:504] T 61d0deb8b2f14981995b7c786d9e48cf P 1671965ee3334946868d67415747aba5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } }
I20260928 14:43:16.880239 17373 leader_election.cc:288] T 61d0deb8b2f14981995b7c786d9e48cf P 1671965ee3334946868d67415747aba5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f8b53fbbb804e189ce714679eb3e35c (127.9.33.66:44523), fb063f3bf2c24ca29f5480118eaa718b (127.9.33.65:33699)
I20260928 14:43:16.883071 17365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:16.880553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:52093 (local address 127.9.33.66:44523)
0928 14:43:16.880597 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:16.880599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:16.880607 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:43:16.880638 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:16.880639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:16.880649 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:16.880708 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:43:16.880910 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.881397 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.882339 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.882700 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.882786 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:16.882797 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:16.882961 (+ 164us) server_negotiation.cc:299] Negotiation successful
0928 14:43:16.883000 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 14:43:16.883085 17232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:16.880458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:44523 (local address 127.9.33.67:52093)
0928 14:43:16.880518 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:16.880530 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:16.880578 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:16.880713 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:16.880713 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:16.880714 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:43:16.880832 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:16.880835 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.881491 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.881492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:16.882243 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:16.882245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.882786 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.882787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:16.882871 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:16.882875 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:43:16.882951 (+ 76us) client_negotiation.cc:240] Negotiation successful
0928 14:43:16.882995 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260928 14:43:16.883152 17366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:16.880524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:45001 (local address 127.9.33.65:33699)
0928 14:43:16.880575 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:16.880577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:16.880720 (+ 143us) server_negotiation.cc:407] Connection header received
0928 14:43:16.880792 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:16.880793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:16.880805 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:16.880857 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:43:16.881028 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.881499 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.882421 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.882787 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.882869 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:16.882880 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:16.883038 (+ 158us) server_negotiation.cc:299] Negotiation successful
0928 14:43:16.883076 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260928 14:43:16.883209 17376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:16.880467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:33699 (local address 127.9.33.67:45001)
0928 14:43:16.880653 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:16.880661 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:16.880736 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:16.880865 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:16.880866 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:16.880868 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:43:16.880941 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:16.880943 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.881589 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.881590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:16.882327 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:16.882329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.882880 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.882882 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:16.882965 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:16.882969 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:43:16.883033 (+ 64us) client_negotiation.cc:240] Negotiation successful
0928 14:43:16.883073 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":3,"thread_start_us":98,"threads_started":1}
I20260928 14:43:16.883447 17153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61d0deb8b2f14981995b7c786d9e48cf" candidate_uuid: "1671965ee3334946868d67415747aba5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f8b53fbbb804e189ce714679eb3e35c" is_pre_election: true
I20260928 14:43:16.883582 17153 raft_consensus.cc:2454] T 61d0deb8b2f14981995b7c786d9e48cf P 7f8b53fbbb804e189ce714679eb3e35c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1671965ee3334946868d67415747aba5 in term 0.
I20260928 14:43:16.883823 17225 leader_election.cc:302] T 61d0deb8b2f14981995b7c786d9e48cf P 1671965ee3334946868d67415747aba5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1671965ee3334946868d67415747aba5, 7f8b53fbbb804e189ce714679eb3e35c; no voters:
I20260928 14:43:16.883867 17013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61d0deb8b2f14981995b7c786d9e48cf" candidate_uuid: "1671965ee3334946868d67415747aba5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb063f3bf2c24ca29f5480118eaa718b" is_pre_election: true
I20260928 14:43:16.883956 17373 raft_consensus.cc:2781] T 61d0deb8b2f14981995b7c786d9e48cf P 1671965ee3334946868d67415747aba5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:43:16.884014 17373 raft_consensus.cc:482] T 61d0deb8b2f14981995b7c786d9e48cf P 1671965ee3334946868d67415747aba5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:43:16.884043 17373 raft_consensus.cc:3037] T 61d0deb8b2f14981995b7c786d9e48cf P 1671965ee3334946868d67415747aba5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:16.884052 17013 raft_consensus.cc:2454] T 61d0deb8b2f14981995b7c786d9e48cf P fb063f3bf2c24ca29f5480118eaa718b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1671965ee3334946868d67415747aba5 in term 0.
I20260928 14:43:16.885087 17373 raft_consensus.cc:504] T 61d0deb8b2f14981995b7c786d9e48cf P 1671965ee3334946868d67415747aba5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } }
I20260928 14:43:16.885245 17373 leader_election.cc:288] T 61d0deb8b2f14981995b7c786d9e48cf P 1671965ee3334946868d67415747aba5 [CANDIDATE]: Term 1 election: Requested vote from peers 7f8b53fbbb804e189ce714679eb3e35c (127.9.33.66:44523), fb063f3bf2c24ca29f5480118eaa718b (127.9.33.65:33699)
I20260928 14:43:16.885450 17013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61d0deb8b2f14981995b7c786d9e48cf" candidate_uuid: "1671965ee3334946868d67415747aba5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb063f3bf2c24ca29f5480118eaa718b"
I20260928 14:43:16.885542 17013 raft_consensus.cc:3037] T 61d0deb8b2f14981995b7c786d9e48cf P fb063f3bf2c24ca29f5480118eaa718b [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:16.885551 17153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61d0deb8b2f14981995b7c786d9e48cf" candidate_uuid: "1671965ee3334946868d67415747aba5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f8b53fbbb804e189ce714679eb3e35c"
I20260928 14:43:16.885640 17153 raft_consensus.cc:3037] T 61d0deb8b2f14981995b7c786d9e48cf P 7f8b53fbbb804e189ce714679eb3e35c [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:16.886230 17013 raft_consensus.cc:2454] T 61d0deb8b2f14981995b7c786d9e48cf P fb063f3bf2c24ca29f5480118eaa718b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1671965ee3334946868d67415747aba5 in term 1.
I20260928 14:43:16.886557 17153 raft_consensus.cc:2454] T 61d0deb8b2f14981995b7c786d9e48cf P 7f8b53fbbb804e189ce714679eb3e35c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1671965ee3334946868d67415747aba5 in term 1.
I20260928 14:43:16.886739 17226 leader_election.cc:302] T 61d0deb8b2f14981995b7c786d9e48cf P 1671965ee3334946868d67415747aba5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1671965ee3334946868d67415747aba5, fb063f3bf2c24ca29f5480118eaa718b; no voters:
I20260928 14:43:16.886857 17373 raft_consensus.cc:2781] T 61d0deb8b2f14981995b7c786d9e48cf P 1671965ee3334946868d67415747aba5 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:43:16.887053 17373 raft_consensus.cc:686] T 61d0deb8b2f14981995b7c786d9e48cf P 1671965ee3334946868d67415747aba5 [term 1 LEADER]: Becoming Leader. State: Replica: 1671965ee3334946868d67415747aba5, State: Running, Role: LEADER
I20260928 14:43:16.887156 17373 consensus_queue.cc:237] T 61d0deb8b2f14981995b7c786d9e48cf P 1671965ee3334946868d67415747aba5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } }
I20260928 14:43:16.888119 16855 catalog_manager.cc:5495] T 61d0deb8b2f14981995b7c786d9e48cf P 1671965ee3334946868d67415747aba5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1671965ee3334946868d67415747aba5 (127.9.33.67). New cstate: current_term: 1 leader_uuid: "1671965ee3334946868d67415747aba5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } health_report { overall_health: UNKNOWN } } }
I20260928 14:43:16.907143 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:16.904683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36422 (local address 127.9.33.67:35545)
0928 14:43:16.904750 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:16.904752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:16.904762 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:43:16.904840 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:16.904841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:16.904861 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:16.904925 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:43:16.905103 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.905667 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.906428 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:16.906617 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:16.906672 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:16.906847 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:43:16.906954 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:43:16.907023 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:16.907064 (+ 41us) server_negotiation.cc:299] Negotiation successful
0928 14:43:16.907082 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260928 14:43:16.909157 17013 raft_consensus.cc:1261] T 61d0deb8b2f14981995b7c786d9e48cf P fb063f3bf2c24ca29f5480118eaa718b [term 1 FOLLOWER]: Refusing update from remote peer 1671965ee3334946868d67415747aba5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:43:16.909155 17153 raft_consensus.cc:1261] T 61d0deb8b2f14981995b7c786d9e48cf P 7f8b53fbbb804e189ce714679eb3e35c [term 1 FOLLOWER]: Refusing update from remote peer 1671965ee3334946868d67415747aba5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:43:16.909528 17377 consensus_queue.cc:1035] T 61d0deb8b2f14981995b7c786d9e48cf P 1671965ee3334946868d67415747aba5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:43:16.909675 17373 consensus_queue.cc:1035] T 61d0deb8b2f14981995b7c786d9e48cf P 1671965ee3334946868d67415747aba5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:43:16.912385 17381 mvcc.cc:205] Tried to move back new op lower bound from 7334324620936830976 to 7334324620850683904. Current Snapshot: MvccSnapshot[applied={T|T < 7334324620936830976}]
I20260928 14:43:16.913173 17382 mvcc.cc:205] Tried to move back new op lower bound from 7334324620936830976 to 7334324620850683904. Current Snapshot: MvccSnapshot[applied={T|T < 7334324620936830976}]
I20260928 14:43:16.914584 16856 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:43:16.914680 16856 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:43:16.915504 16856 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0c41cc0a857d4203a8750d8a6b304591: Sending DeleteTablet for 3 replicas of tablet 61d0deb8b2f14981995b7c786d9e48cf
I20260928 14:43:16.915856 17273 tablet_service.cc:1514] Processing DeleteTablet for tablet 61d0deb8b2f14981995b7c786d9e48cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:43:16 UTC) from {username='oryx', principal='oryx/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:36406
I20260928 14:43:16.915920 17133 tablet_service.cc:1514] Processing DeleteTablet for tablet 61d0deb8b2f14981995b7c786d9e48cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:43:16 UTC) from {username='oryx', principal='oryx/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:39600
I20260928 14:43:16.916147 17389 tablet_replica.cc:330] T 61d0deb8b2f14981995b7c786d9e48cf P 1671965ee3334946868d67415747aba5: stopping tablet replica
I20260928 14:43:16.916170 16993 tablet_service.cc:1514] Processing DeleteTablet for tablet 61d0deb8b2f14981995b7c786d9e48cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:43:16 UTC) from {username='oryx', principal='oryx/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:54694
I20260928 14:43:16.916247 17389 raft_consensus.cc:2229] T 61d0deb8b2f14981995b7c786d9e48cf P 1671965ee3334946868d67415747aba5 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:43:16.916347 17391 tablet_replica.cc:330] T 61d0deb8b2f14981995b7c786d9e48cf P fb063f3bf2c24ca29f5480118eaa718b: stopping tablet replica
I20260928 14:43:16.916352 17389 raft_consensus.cc:2258] T 61d0deb8b2f14981995b7c786d9e48cf P 1671965ee3334946868d67415747aba5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:43:16.916538 17391 raft_consensus.cc:2229] T 61d0deb8b2f14981995b7c786d9e48cf P fb063f3bf2c24ca29f5480118eaa718b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:43:16.916471 16856 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46696:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:43:16.916610 16856 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:43:16.916643 17391 raft_consensus.cc:2258] T 61d0deb8b2f14981995b7c786d9e48cf P fb063f3bf2c24ca29f5480118eaa718b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:43:16.916792 17389 ts_tablet_manager.cc:1905] T 61d0deb8b2f14981995b7c786d9e48cf P 1671965ee3334946868d67415747aba5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:43:16.917117 17391 ts_tablet_manager.cc:1905] T 61d0deb8b2f14981995b7c786d9e48cf P fb063f3bf2c24ca29f5480118eaa718b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:43:16.918040 17273 tablet_service.cc:1467] Processing CreateTablet for tablet ca042f873a35462d98b61c5fefcc58d4 (DEFAULT_TABLE table=test-table [id=24f6c19f1a8942b089260ee1fdfb8ce5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:43:16.918152 17273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca042f873a35462d98b61c5fefcc58d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:16.918722 17389 ts_tablet_manager.cc:1918] T 61d0deb8b2f14981995b7c786d9e48cf P 1671965ee3334946868d67415747aba5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:43:16.918789 17389 log.cc:1196] T 61d0deb8b2f14981995b7c786d9e48cf P 1671965ee3334946868d67415747aba5: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-2/wal/wals/61d0deb8b2f14981995b7c786d9e48cf
I20260928 14:43:16.918849 17391 ts_tablet_manager.cc:1918] T 61d0deb8b2f14981995b7c786d9e48cf P fb063f3bf2c24ca29f5480118eaa718b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:43:16.918934 17391 log.cc:1196] T 61d0deb8b2f14981995b7c786d9e48cf P fb063f3bf2c24ca29f5480118eaa718b: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-0/wal/wals/61d0deb8b2f14981995b7c786d9e48cf
I20260928 14:43:16.919049 17389 ts_tablet_manager.cc:1939] T 61d0deb8b2f14981995b7c786d9e48cf P 1671965ee3334946868d67415747aba5: Deleting consensus metadata
I20260928 14:43:16.919212 17391 ts_tablet_manager.cc:1939] T 61d0deb8b2f14981995b7c786d9e48cf P fb063f3bf2c24ca29f5480118eaa718b: Deleting consensus metadata
I20260928 14:43:16.919466 16840 catalog_manager.cc:4841] TS 1671965ee3334946868d67415747aba5 (127.9.33.67:35545): tablet 61d0deb8b2f14981995b7c786d9e48cf (table test-table [id=cf2ee29f1f9545dc89d131fe8b910bca]) successfully deleted
I20260928 14:43:16.919585 17367 tablet_bootstrap.cc:492] T ca042f873a35462d98b61c5fefcc58d4 P 1671965ee3334946868d67415747aba5: Bootstrap starting.
I20260928 14:43:16.920032 17367 tablet_bootstrap.cc:654] T ca042f873a35462d98b61c5fefcc58d4 P 1671965ee3334946868d67415747aba5: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:16.920029 17133 tablet_service.cc:1467] Processing CreateTablet for tablet ca042f873a35462d98b61c5fefcc58d4 (DEFAULT_TABLE table=test-table [id=24f6c19f1a8942b089260ee1fdfb8ce5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:43:16.920148 17133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca042f873a35462d98b61c5fefcc58d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:16.920764 17367 tablet_bootstrap.cc:492] T ca042f873a35462d98b61c5fefcc58d4 P 1671965ee3334946868d67415747aba5: No bootstrap required, opened a new log
I20260928 14:43:16.920819 17367 ts_tablet_manager.cc:1397] T ca042f873a35462d98b61c5fefcc58d4 P 1671965ee3334946868d67415747aba5: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260928 14:43:16.920910 17390 tablet_replica.cc:330] T 61d0deb8b2f14981995b7c786d9e48cf P 7f8b53fbbb804e189ce714679eb3e35c: stopping tablet replica
I20260928 14:43:16.920967 17367 raft_consensus.cc:348] T ca042f873a35462d98b61c5fefcc58d4 P 1671965ee3334946868d67415747aba5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } }
I20260928 14:43:16.921049 17367 raft_consensus.cc:374] T ca042f873a35462d98b61c5fefcc58d4 P 1671965ee3334946868d67415747aba5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:16.921077 17367 raft_consensus.cc:729] T ca042f873a35462d98b61c5fefcc58d4 P 1671965ee3334946868d67415747aba5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1671965ee3334946868d67415747aba5, State: Initialized, Role: FOLLOWER
I20260928 14:43:16.921118 17367 consensus_queue.cc:260] T ca042f873a35462d98b61c5fefcc58d4 P 1671965ee3334946868d67415747aba5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } }
I20260928 14:43:16.921154 17390 raft_consensus.cc:2229] T 61d0deb8b2f14981995b7c786d9e48cf P 7f8b53fbbb804e189ce714679eb3e35c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:43:16.921222 17367 ts_tablet_manager.cc:1428] T ca042f873a35462d98b61c5fefcc58d4 P 1671965ee3334946868d67415747aba5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:16.921396 17390 raft_consensus.cc:2258] T 61d0deb8b2f14981995b7c786d9e48cf P 7f8b53fbbb804e189ce714679eb3e35c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:43:16.921640 16993 tablet_service.cc:1467] Processing CreateTablet for tablet ca042f873a35462d98b61c5fefcc58d4 (DEFAULT_TABLE table=test-table [id=24f6c19f1a8942b089260ee1fdfb8ce5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:43:16.921764 16993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca042f873a35462d98b61c5fefcc58d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:16.922912 17368 tablet_bootstrap.cc:492] T ca042f873a35462d98b61c5fefcc58d4 P fb063f3bf2c24ca29f5480118eaa718b: Bootstrap starting.
I20260928 14:43:16.923054 17371 tablet_bootstrap.cc:492] T ca042f873a35462d98b61c5fefcc58d4 P 7f8b53fbbb804e189ce714679eb3e35c: Bootstrap starting.
I20260928 14:43:16.923336 17368 tablet_bootstrap.cc:654] T ca042f873a35462d98b61c5fefcc58d4 P fb063f3bf2c24ca29f5480118eaa718b: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:16.923498 17371 tablet_bootstrap.cc:654] T ca042f873a35462d98b61c5fefcc58d4 P 7f8b53fbbb804e189ce714679eb3e35c: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:16.924152 17371 tablet_bootstrap.cc:492] T ca042f873a35462d98b61c5fefcc58d4 P 7f8b53fbbb804e189ce714679eb3e35c: No bootstrap required, opened a new log
I20260928 14:43:16.924180 17390 ts_tablet_manager.cc:1905] T 61d0deb8b2f14981995b7c786d9e48cf P 7f8b53fbbb804e189ce714679eb3e35c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:43:16.924226 17371 ts_tablet_manager.cc:1397] T ca042f873a35462d98b61c5fefcc58d4 P 7f8b53fbbb804e189ce714679eb3e35c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:16.924396 17371 raft_consensus.cc:348] T ca042f873a35462d98b61c5fefcc58d4 P 7f8b53fbbb804e189ce714679eb3e35c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } }
I20260928 14:43:16.924485 17371 raft_consensus.cc:374] T ca042f873a35462d98b61c5fefcc58d4 P 7f8b53fbbb804e189ce714679eb3e35c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:16.924513 17371 raft_consensus.cc:729] T ca042f873a35462d98b61c5fefcc58d4 P 7f8b53fbbb804e189ce714679eb3e35c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f8b53fbbb804e189ce714679eb3e35c, State: Initialized, Role: FOLLOWER
I20260928 14:43:16.924559 17371 consensus_queue.cc:260] T ca042f873a35462d98b61c5fefcc58d4 P 7f8b53fbbb804e189ce714679eb3e35c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } }
I20260928 14:43:16.924656 17371 ts_tablet_manager.cc:1428] T ca042f873a35462d98b61c5fefcc58d4 P 7f8b53fbbb804e189ce714679eb3e35c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260928 14:43:16.926003 17390 ts_tablet_manager.cc:1918] T 61d0deb8b2f14981995b7c786d9e48cf P 7f8b53fbbb804e189ce714679eb3e35c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:43:16.926018 17368 tablet_bootstrap.cc:492] T ca042f873a35462d98b61c5fefcc58d4 P fb063f3bf2c24ca29f5480118eaa718b: No bootstrap required, opened a new log
I20260928 14:43:16.926069 17368 ts_tablet_manager.cc:1397] T ca042f873a35462d98b61c5fefcc58d4 P fb063f3bf2c24ca29f5480118eaa718b: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260928 14:43:16.926085 17390 log.cc:1196] T 61d0deb8b2f14981995b7c786d9e48cf P 7f8b53fbbb804e189ce714679eb3e35c: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-1/wal/wals/61d0deb8b2f14981995b7c786d9e48cf
I20260928 14:43:16.926378 17390 ts_tablet_manager.cc:1939] T 61d0deb8b2f14981995b7c786d9e48cf P 7f8b53fbbb804e189ce714679eb3e35c: Deleting consensus metadata
I20260928 14:43:16.926451 17368 raft_consensus.cc:348] T ca042f873a35462d98b61c5fefcc58d4 P fb063f3bf2c24ca29f5480118eaa718b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } }
I20260928 14:43:16.926528 17368 raft_consensus.cc:374] T ca042f873a35462d98b61c5fefcc58d4 P fb063f3bf2c24ca29f5480118eaa718b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:16.926549 17368 raft_consensus.cc:729] T ca042f873a35462d98b61c5fefcc58d4 P fb063f3bf2c24ca29f5480118eaa718b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb063f3bf2c24ca29f5480118eaa718b, State: Initialized, Role: FOLLOWER
I20260928 14:43:16.926599 17368 consensus_queue.cc:260] T ca042f873a35462d98b61c5fefcc58d4 P fb063f3bf2c24ca29f5480118eaa718b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } }
I20260928 14:43:16.926707 17368 ts_tablet_manager.cc:1428] T ca042f873a35462d98b61c5fefcc58d4 P fb063f3bf2c24ca29f5480118eaa718b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260928 14:43:16.926775 16841 catalog_manager.cc:4841] TS 7f8b53fbbb804e189ce714679eb3e35c (127.9.33.66:44523): tablet 61d0deb8b2f14981995b7c786d9e48cf (table test-table [id=cf2ee29f1f9545dc89d131fe8b910bca]) successfully deleted
I20260928 14:43:16.926976 16842 catalog_manager.cc:4841] TS fb063f3bf2c24ca29f5480118eaa718b (127.9.33.65:33699): tablet 61d0deb8b2f14981995b7c786d9e48cf (table test-table [id=cf2ee29f1f9545dc89d131fe8b910bca]) successfully deleted
W20260928 14:43:16.958166 17340 tablet.cc:2369] T ca042f873a35462d98b61c5fefcc58d4 P 1671965ee3334946868d67415747aba5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:43:17.018899 17377 raft_consensus.cc:482] T ca042f873a35462d98b61c5fefcc58d4 P 1671965ee3334946868d67415747aba5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:43:17.019029 17377 raft_consensus.cc:504] T ca042f873a35462d98b61c5fefcc58d4 P 1671965ee3334946868d67415747aba5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } }
I20260928 14:43:17.019253 17377 leader_election.cc:288] T ca042f873a35462d98b61c5fefcc58d4 P 1671965ee3334946868d67415747aba5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb063f3bf2c24ca29f5480118eaa718b (127.9.33.65:33699), 7f8b53fbbb804e189ce714679eb3e35c (127.9.33.66:44523)
I20260928 14:43:17.019464 17013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca042f873a35462d98b61c5fefcc58d4" candidate_uuid: "1671965ee3334946868d67415747aba5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb063f3bf2c24ca29f5480118eaa718b" is_pre_election: true
I20260928 14:43:17.019464 17153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca042f873a35462d98b61c5fefcc58d4" candidate_uuid: "1671965ee3334946868d67415747aba5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f8b53fbbb804e189ce714679eb3e35c" is_pre_election: true
I20260928 14:43:17.019567 17153 raft_consensus.cc:2454] T ca042f873a35462d98b61c5fefcc58d4 P 7f8b53fbbb804e189ce714679eb3e35c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1671965ee3334946868d67415747aba5 in term 0.
I20260928 14:43:17.019567 17013 raft_consensus.cc:2454] T ca042f873a35462d98b61c5fefcc58d4 P fb063f3bf2c24ca29f5480118eaa718b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1671965ee3334946868d67415747aba5 in term 0.
I20260928 14:43:17.019742 17225 leader_election.cc:302] T ca042f873a35462d98b61c5fefcc58d4 P 1671965ee3334946868d67415747aba5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1671965ee3334946868d67415747aba5, 7f8b53fbbb804e189ce714679eb3e35c; no voters:
I20260928 14:43:17.019866 17377 raft_consensus.cc:2781] T ca042f873a35462d98b61c5fefcc58d4 P 1671965ee3334946868d67415747aba5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:43:17.019933 17377 raft_consensus.cc:482] T ca042f873a35462d98b61c5fefcc58d4 P 1671965ee3334946868d67415747aba5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:43:17.019966 17377 raft_consensus.cc:3037] T ca042f873a35462d98b61c5fefcc58d4 P 1671965ee3334946868d67415747aba5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:17.020738 17377 raft_consensus.cc:504] T ca042f873a35462d98b61c5fefcc58d4 P 1671965ee3334946868d67415747aba5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } }
I20260928 14:43:17.020896 17377 leader_election.cc:288] T ca042f873a35462d98b61c5fefcc58d4 P 1671965ee3334946868d67415747aba5 [CANDIDATE]: Term 1 election: Requested vote from peers fb063f3bf2c24ca29f5480118eaa718b (127.9.33.65:33699), 7f8b53fbbb804e189ce714679eb3e35c (127.9.33.66:44523)
I20260928 14:43:17.021059 17153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca042f873a35462d98b61c5fefcc58d4" candidate_uuid: "1671965ee3334946868d67415747aba5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f8b53fbbb804e189ce714679eb3e35c"
I20260928 14:43:17.021145 17153 raft_consensus.cc:3037] T ca042f873a35462d98b61c5fefcc58d4 P 7f8b53fbbb804e189ce714679eb3e35c [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:17.021144 17013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca042f873a35462d98b61c5fefcc58d4" candidate_uuid: "1671965ee3334946868d67415747aba5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb063f3bf2c24ca29f5480118eaa718b"
I20260928 14:43:17.021229 17013 raft_consensus.cc:3037] T ca042f873a35462d98b61c5fefcc58d4 P fb063f3bf2c24ca29f5480118eaa718b [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:17.021973 17153 raft_consensus.cc:2454] T ca042f873a35462d98b61c5fefcc58d4 P 7f8b53fbbb804e189ce714679eb3e35c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1671965ee3334946868d67415747aba5 in term 1.
I20260928 14:43:17.021974 17013 raft_consensus.cc:2454] T ca042f873a35462d98b61c5fefcc58d4 P fb063f3bf2c24ca29f5480118eaa718b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1671965ee3334946868d67415747aba5 in term 1.
I20260928 14:43:17.022166 17225 leader_election.cc:302] T ca042f873a35462d98b61c5fefcc58d4 P 1671965ee3334946868d67415747aba5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1671965ee3334946868d67415747aba5, 7f8b53fbbb804e189ce714679eb3e35c, fb063f3bf2c24ca29f5480118eaa718b; no voters:
I20260928 14:43:17.022261 17377 raft_consensus.cc:2781] T ca042f873a35462d98b61c5fefcc58d4 P 1671965ee3334946868d67415747aba5 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:43:17.022312 17377 raft_consensus.cc:686] T ca042f873a35462d98b61c5fefcc58d4 P 1671965ee3334946868d67415747aba5 [term 1 LEADER]: Becoming Leader. State: Replica: 1671965ee3334946868d67415747aba5, State: Running, Role: LEADER
I20260928 14:43:17.022365 17377 consensus_queue.cc:237] T ca042f873a35462d98b61c5fefcc58d4 P 1671965ee3334946868d67415747aba5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } }
I20260928 14:43:17.023044 16856 catalog_manager.cc:5495] T ca042f873a35462d98b61c5fefcc58d4 P 1671965ee3334946868d67415747aba5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1671965ee3334946868d67415747aba5 (127.9.33.67). New cstate: current_term: 1 leader_uuid: "1671965ee3334946868d67415747aba5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } health_report { overall_health: HEALTHY } } }
Sep 28 14:43:17 dist-test-slave-pv0w krb5kdc[16811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606596, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.126@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM
I20260928 14:43:17.079929 16905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:17.072646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46712 (local address 127.9.33.126:36113)
0928 14:43:17.072722 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:17.072724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:17.072865 (+ 141us) server_negotiation.cc:407] Connection header received
0928 14:43:17.072911 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:17.072912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:17.072922 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:17.072975 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:17.073441 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:17.073935 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:17.074635 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:17.074837 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:17.074893 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:17.077166 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:17.077168 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:17.077168 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:17.077176 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:17.078673 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:17.079056 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:17.079057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:17.079058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:17.079114 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:17.079315 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:17.079315 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:17.079316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:17.079429 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:17.079502 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:17.079673 (+ 171us) server_negotiation.cc:299] Negotiation successful
0928 14:43:17.079814 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260928 14:43:17.080003 17401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:17.072594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:36113 (local address 127.0.0.1:46712)
0928 14:43:17.072771 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:17.072782 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:17.072861 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:17.072992 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:17.072993 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:17.073319 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:17.073371 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:17.073373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:17.074005 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:17.074006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:17.074554 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:17.074555 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:17.074907 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:17.074907 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:17.074951 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:17.075488 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:17.075493 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:17.077070 (+ 1577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:17.078760 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:17.078768 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:17.078771 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:17.078979 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:17.079181 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:17.079183 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:17.079183 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:17.079248 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:17.079504 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:17.079518 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:17.079564 (+ 46us) client_negotiation.cc:769] Sending connection context
0928 14:43:17.079650 (+ 86us) client_negotiation.cc:240] Negotiation successful
0928 14:43:17.079877 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":121,"thread_start_us":108,"threads_started":1}
I20260928 14:43:17.082118 16856 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:46712:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260928 14:43:17.082309 16856 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:43:17.083959 16993 tablet_service.cc:1467] Processing CreateTablet for tablet 4a7aa3fe12e84eeca3f5297742f3f779 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1d46e7b8819a40d88470409454ea88aa]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 14:43:17.083966 17273 tablet_service.cc:1467] Processing CreateTablet for tablet 4a7aa3fe12e84eeca3f5297742f3f779 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1d46e7b8819a40d88470409454ea88aa]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 14:43:17.084137 17273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a7aa3fe12e84eeca3f5297742f3f779. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:17.084131 16993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a7aa3fe12e84eeca3f5297742f3f779. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:17.085246 17367 tablet_bootstrap.cc:492] T 4a7aa3fe12e84eeca3f5297742f3f779 P 1671965ee3334946868d67415747aba5: Bootstrap starting.
I20260928 14:43:17.085407 17133 tablet_service.cc:1467] Processing CreateTablet for tablet 4a7aa3fe12e84eeca3f5297742f3f779 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1d46e7b8819a40d88470409454ea88aa]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260928 14:43:17.085552 17133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a7aa3fe12e84eeca3f5297742f3f779. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:17.085630 17367 tablet_bootstrap.cc:654] T 4a7aa3fe12e84eeca3f5297742f3f779 P 1671965ee3334946868d67415747aba5: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:17.086274 17367 tablet_bootstrap.cc:492] T 4a7aa3fe12e84eeca3f5297742f3f779 P 1671965ee3334946868d67415747aba5: No bootstrap required, opened a new log
I20260928 14:43:17.086325 17367 ts_tablet_manager.cc:1397] T 4a7aa3fe12e84eeca3f5297742f3f779 P 1671965ee3334946868d67415747aba5: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260928 14:43:17.086450 17367 raft_consensus.cc:348] T 4a7aa3fe12e84eeca3f5297742f3f779 P 1671965ee3334946868d67415747aba5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } }
I20260928 14:43:17.086524 17367 raft_consensus.cc:374] T 4a7aa3fe12e84eeca3f5297742f3f779 P 1671965ee3334946868d67415747aba5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:17.086539 17367 raft_consensus.cc:729] T 4a7aa3fe12e84eeca3f5297742f3f779 P 1671965ee3334946868d67415747aba5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1671965ee3334946868d67415747aba5, State: Initialized, Role: FOLLOWER
I20260928 14:43:17.086593 17367 consensus_queue.cc:260] T 4a7aa3fe12e84eeca3f5297742f3f779 P 1671965ee3334946868d67415747aba5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } }
I20260928 14:43:17.086699 17367 ts_tablet_manager.cc:1428] T 4a7aa3fe12e84eeca3f5297742f3f779 P 1671965ee3334946868d67415747aba5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:17.086715 17377 tablet_replica.cc:439] T 4a7aa3fe12e84eeca3f5297742f3f779 P 1671965ee3334946868d67415747aba5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } } }
I20260928 14:43:17.086797 17377 tablet_replica.cc:442] T 4a7aa3fe12e84eeca3f5297742f3f779 P 1671965ee3334946868d67415747aba5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:43:17.087002 17371 tablet_bootstrap.cc:492] T 4a7aa3fe12e84eeca3f5297742f3f779 P 7f8b53fbbb804e189ce714679eb3e35c: Bootstrap starting.
I20260928 14:43:17.087051 17368 tablet_bootstrap.cc:492] T 4a7aa3fe12e84eeca3f5297742f3f779 P fb063f3bf2c24ca29f5480118eaa718b: Bootstrap starting.
I20260928 14:43:17.087435 17371 tablet_bootstrap.cc:654] T 4a7aa3fe12e84eeca3f5297742f3f779 P 7f8b53fbbb804e189ce714679eb3e35c: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:17.087456 17368 tablet_bootstrap.cc:654] T 4a7aa3fe12e84eeca3f5297742f3f779 P fb063f3bf2c24ca29f5480118eaa718b: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:17.088124 17371 tablet_bootstrap.cc:492] T 4a7aa3fe12e84eeca3f5297742f3f779 P 7f8b53fbbb804e189ce714679eb3e35c: No bootstrap required, opened a new log
I20260928 14:43:17.088178 17371 ts_tablet_manager.cc:1397] T 4a7aa3fe12e84eeca3f5297742f3f779 P 7f8b53fbbb804e189ce714679eb3e35c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260928 14:43:17.088325 17368 tablet_bootstrap.cc:492] T 4a7aa3fe12e84eeca3f5297742f3f779 P fb063f3bf2c24ca29f5480118eaa718b: No bootstrap required, opened a new log
I20260928 14:43:17.088336 17371 raft_consensus.cc:348] T 4a7aa3fe12e84eeca3f5297742f3f779 P 7f8b53fbbb804e189ce714679eb3e35c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } }
I20260928 14:43:17.088389 17368 ts_tablet_manager.cc:1397] T 4a7aa3fe12e84eeca3f5297742f3f779 P fb063f3bf2c24ca29f5480118eaa718b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260928 14:43:17.088388 17371 raft_consensus.cc:374] T 4a7aa3fe12e84eeca3f5297742f3f779 P 7f8b53fbbb804e189ce714679eb3e35c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:17.088428 17371 raft_consensus.cc:729] T 4a7aa3fe12e84eeca3f5297742f3f779 P 7f8b53fbbb804e189ce714679eb3e35c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f8b53fbbb804e189ce714679eb3e35c, State: Initialized, Role: FOLLOWER
I20260928 14:43:17.088481 17371 consensus_queue.cc:260] T 4a7aa3fe12e84eeca3f5297742f3f779 P 7f8b53fbbb804e189ce714679eb3e35c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } }
I20260928 14:43:17.088585 17371 ts_tablet_manager.cc:1428] T 4a7aa3fe12e84eeca3f5297742f3f779 P 7f8b53fbbb804e189ce714679eb3e35c: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260928 14:43:17.088549 17368 raft_consensus.cc:348] T 4a7aa3fe12e84eeca3f5297742f3f779 P fb063f3bf2c24ca29f5480118eaa718b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } }
I20260928 14:43:17.088620 17368 raft_consensus.cc:374] T 4a7aa3fe12e84eeca3f5297742f3f779 P fb063f3bf2c24ca29f5480118eaa718b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:17.088649 17368 raft_consensus.cc:729] T 4a7aa3fe12e84eeca3f5297742f3f779 P fb063f3bf2c24ca29f5480118eaa718b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb063f3bf2c24ca29f5480118eaa718b, State: Initialized, Role: FOLLOWER
I20260928 14:43:17.088631 17375 tablet_replica.cc:439] T 4a7aa3fe12e84eeca3f5297742f3f779 P 7f8b53fbbb804e189ce714679eb3e35c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } } }
I20260928 14:43:17.088704 17375 tablet_replica.cc:442] T 4a7aa3fe12e84eeca3f5297742f3f779 P 7f8b53fbbb804e189ce714679eb3e35c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:43:17.088698 17368 consensus_queue.cc:260] T 4a7aa3fe12e84eeca3f5297742f3f779 P fb063f3bf2c24ca29f5480118eaa718b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } }
I20260928 14:43:17.088804 17368 ts_tablet_manager.cc:1428] T 4a7aa3fe12e84eeca3f5297742f3f779 P fb063f3bf2c24ca29f5480118eaa718b: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260928 14:43:17.088973 17374 tablet_replica.cc:439] T 4a7aa3fe12e84eeca3f5297742f3f779 P fb063f3bf2c24ca29f5480118eaa718b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } } }
I20260928 14:43:17.089075 17374 tablet_replica.cc:442] T 4a7aa3fe12e84eeca3f5297742f3f779 P fb063f3bf2c24ca29f5480118eaa718b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:43:17.211122 17374 raft_consensus.cc:482] T 4a7aa3fe12e84eeca3f5297742f3f779 P fb063f3bf2c24ca29f5480118eaa718b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:43:17.211231 17374 raft_consensus.cc:504] T 4a7aa3fe12e84eeca3f5297742f3f779 P fb063f3bf2c24ca29f5480118eaa718b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } }
I20260928 14:43:17.211772 17374 leader_election.cc:288] T 4a7aa3fe12e84eeca3f5297742f3f779 P fb063f3bf2c24ca29f5480118eaa718b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f8b53fbbb804e189ce714679eb3e35c (127.9.33.66:44523), 1671965ee3334946868d67415747aba5 (127.9.33.67:35545)
I20260928 14:43:17.215112 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:17.212097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:37103 (local address 127.9.33.67:35545)
0928 14:43:17.212147 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:17.212149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:17.212315 (+ 166us) server_negotiation.cc:407] Connection header received
0928 14:43:17.212376 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:17.212377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:17.212388 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:17.212443 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:43:17.212617 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:17.213205 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:17.214326 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:17.214703 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:17.214812 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:17.214823 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:17.214996 (+ 173us) server_negotiation.cc:299] Negotiation successful
0928 14:43:17.215035 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 14:43:17.215116 17365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:17.212069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:55483 (local address 127.9.33.66:44523)
0928 14:43:17.212121 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:17.212123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:17.212389 (+ 266us) server_negotiation.cc:407] Connection header received
0928 14:43:17.212445 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:17.212446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:17.212456 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:17.212511 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:43:17.212729 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:17.213317 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:17.214320 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:17.214698 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:17.214807 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:17.214819 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:17.215009 (+ 190us) server_negotiation.cc:299] Negotiation successful
0928 14:43:17.215049 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260928 14:43:17.215142 17405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:17.212054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:35545 (local address 127.9.33.65:37103)
0928 14:43:17.212240 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:17.212252 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:17.212321 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:17.212444 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:17.212445 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:17.212447 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:43:17.212541 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:17.212543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:17.213313 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:17.213314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:17.214213 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:17.214216 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:17.214807 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:17.214808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:17.214911 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:17.214915 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:43:17.214988 (+ 73us) client_negotiation.cc:240] Negotiation successful
0928 14:43:17.215035 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":30,"thread_start_us":146,"threads_started":1}
I20260928 14:43:17.215144 17406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:17.212054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:44523 (local address 127.9.33.65:55483)
0928 14:43:17.212270 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:17.212281 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:17.212377 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:17.212524 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:17.212525 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:17.212527 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:43:17.212665 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:17.212667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:17.213405 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:17.213406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:17.214214 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:17.214216 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:17.214811 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:17.214812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:17.214926 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:17.214930 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:43:17.215000 (+ 70us) client_negotiation.cc:240] Negotiation successful
0928 14:43:17.215046 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":40,"thread_start_us":107,"threads_started":1}
I20260928 14:43:17.215485 17293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a7aa3fe12e84eeca3f5297742f3f779" candidate_uuid: "fb063f3bf2c24ca29f5480118eaa718b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1671965ee3334946868d67415747aba5" is_pre_election: true
I20260928 14:43:17.215626 17293 raft_consensus.cc:2454] T 4a7aa3fe12e84eeca3f5297742f3f779 P 1671965ee3334946868d67415747aba5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb063f3bf2c24ca29f5480118eaa718b in term 0.
I20260928 14:43:17.215685 17153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a7aa3fe12e84eeca3f5297742f3f779" candidate_uuid: "fb063f3bf2c24ca29f5480118eaa718b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f8b53fbbb804e189ce714679eb3e35c" is_pre_election: true
I20260928 14:43:17.215811 17153 raft_consensus.cc:2454] T 4a7aa3fe12e84eeca3f5297742f3f779 P 7f8b53fbbb804e189ce714679eb3e35c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb063f3bf2c24ca29f5480118eaa718b in term 0.
I20260928 14:43:17.215868 16944 leader_election.cc:302] T 4a7aa3fe12e84eeca3f5297742f3f779 P fb063f3bf2c24ca29f5480118eaa718b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1671965ee3334946868d67415747aba5, fb063f3bf2c24ca29f5480118eaa718b; no voters:
I20260928 14:43:17.216043 17374 raft_consensus.cc:2781] T 4a7aa3fe12e84eeca3f5297742f3f779 P fb063f3bf2c24ca29f5480118eaa718b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:43:17.216104 17374 raft_consensus.cc:482] T 4a7aa3fe12e84eeca3f5297742f3f779 P fb063f3bf2c24ca29f5480118eaa718b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:43:17.216135 17374 raft_consensus.cc:3037] T 4a7aa3fe12e84eeca3f5297742f3f779 P fb063f3bf2c24ca29f5480118eaa718b [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:17.216895 17374 raft_consensus.cc:504] T 4a7aa3fe12e84eeca3f5297742f3f779 P fb063f3bf2c24ca29f5480118eaa718b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } }
I20260928 14:43:17.217067 17374 leader_election.cc:288] T 4a7aa3fe12e84eeca3f5297742f3f779 P fb063f3bf2c24ca29f5480118eaa718b [CANDIDATE]: Term 1 election: Requested vote from peers 7f8b53fbbb804e189ce714679eb3e35c (127.9.33.66:44523), 1671965ee3334946868d67415747aba5 (127.9.33.67:35545)
I20260928 14:43:17.217228 17153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a7aa3fe12e84eeca3f5297742f3f779" candidate_uuid: "fb063f3bf2c24ca29f5480118eaa718b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f8b53fbbb804e189ce714679eb3e35c"
I20260928 14:43:17.217317 17153 raft_consensus.cc:3037] T 4a7aa3fe12e84eeca3f5297742f3f779 P 7f8b53fbbb804e189ce714679eb3e35c [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:17.218011 17153 raft_consensus.cc:2454] T 4a7aa3fe12e84eeca3f5297742f3f779 P 7f8b53fbbb804e189ce714679eb3e35c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb063f3bf2c24ca29f5480118eaa718b in term 1.
I20260928 14:43:17.218149 17293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a7aa3fe12e84eeca3f5297742f3f779" candidate_uuid: "fb063f3bf2c24ca29f5480118eaa718b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1671965ee3334946868d67415747aba5"
I20260928 14:43:17.218226 16945 leader_election.cc:302] T 4a7aa3fe12e84eeca3f5297742f3f779 P fb063f3bf2c24ca29f5480118eaa718b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f8b53fbbb804e189ce714679eb3e35c, fb063f3bf2c24ca29f5480118eaa718b; no voters:
I20260928 14:43:17.218245 17293 raft_consensus.cc:3037] T 4a7aa3fe12e84eeca3f5297742f3f779 P 1671965ee3334946868d67415747aba5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:17.218317 17374 raft_consensus.cc:2781] T 4a7aa3fe12e84eeca3f5297742f3f779 P fb063f3bf2c24ca29f5480118eaa718b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:43:17.218497 17374 raft_consensus.cc:686] T 4a7aa3fe12e84eeca3f5297742f3f779 P fb063f3bf2c24ca29f5480118eaa718b [term 1 LEADER]: Becoming Leader. State: Replica: fb063f3bf2c24ca29f5480118eaa718b, State: Running, Role: LEADER
I20260928 14:43:17.218592 17374 consensus_queue.cc:237] T 4a7aa3fe12e84eeca3f5297742f3f779 P fb063f3bf2c24ca29f5480118eaa718b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } }
I20260928 14:43:17.218982 17407 tablet_replica.cc:439] T 4a7aa3fe12e84eeca3f5297742f3f779 P fb063f3bf2c24ca29f5480118eaa718b: TxnStatusTablet state changed. Reason: New leader fb063f3bf2c24ca29f5480118eaa718b. Latest consensus state: current_term: 1 leader_uuid: "fb063f3bf2c24ca29f5480118eaa718b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } } }
I20260928 14:43:17.219115 17407 tablet_replica.cc:442] T 4a7aa3fe12e84eeca3f5297742f3f779 P fb063f3bf2c24ca29f5480118eaa718b: This TxnStatusTablet replica's current role is: LEADER
I20260928 14:43:17.219202 17293 raft_consensus.cc:2454] T 4a7aa3fe12e84eeca3f5297742f3f779 P 1671965ee3334946868d67415747aba5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb063f3bf2c24ca29f5480118eaa718b in term 1.
I20260928 14:43:17.219316 17409 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260928 14:43:17.219461 17409 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260928 14:43:17.219926 16856 catalog_manager.cc:5495] T 4a7aa3fe12e84eeca3f5297742f3f779 P fb063f3bf2c24ca29f5480118eaa718b reported cstate change: term changed from 0 to 1, leader changed from <none> to fb063f3bf2c24ca29f5480118eaa718b (127.9.33.65). New cstate: current_term: 1 leader_uuid: "fb063f3bf2c24ca29f5480118eaa718b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } health_report { overall_health: UNKNOWN } } }
I20260928 14:43:17.238723 17366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:17.236010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54708 (local address 127.9.33.65:33699)
0928 14:43:17.236079 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:17.236082 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:17.236114 (+ 32us) server_negotiation.cc:407] Connection header received
0928 14:43:17.236248 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:17.236249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:17.236277 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:17.236344 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:43:17.236518 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:17.237023 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:17.237822 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:17.238038 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:17.238118 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:17.238322 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:43:17.238470 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:43:17.238545 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:17.238591 (+ 46us) server_negotiation.cc:299] Negotiation successful
0928 14:43:17.238616 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260928 14:43:17.238718 17401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:17.235929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:33699 (local address 127.0.0.1:54708)
0928 14:43:17.236062 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:17.236075 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:17.236136 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:17.236364 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:17.236366 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:17.236367 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 14:43:17.236437 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:17.236452 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:17.237123 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:17.237124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:17.237723 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:17.237724 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:17.238116 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:17.238117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:17.238187 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:17.238215 (+ 28us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 14:43:17.238528 (+ 313us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 14:43:17.238539 (+ 11us) client_negotiation.cc:769] Sending connection context
0928 14:43:17.238589 (+ 50us) client_negotiation.cc:240] Negotiation successful
0928 14:43:17.238642 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260928 14:43:17.240170 17153 raft_consensus.cc:1261] T 4a7aa3fe12e84eeca3f5297742f3f779 P 7f8b53fbbb804e189ce714679eb3e35c [term 1 FOLLOWER]: Refusing update from remote peer fb063f3bf2c24ca29f5480118eaa718b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:43:17.240284 17293 raft_consensus.cc:1261] T 4a7aa3fe12e84eeca3f5297742f3f779 P 1671965ee3334946868d67415747aba5 [term 1 FOLLOWER]: Refusing update from remote peer fb063f3bf2c24ca29f5480118eaa718b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:43:17.240473 17374 consensus_queue.cc:1035] T 4a7aa3fe12e84eeca3f5297742f3f779 P fb063f3bf2c24ca29f5480118eaa718b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:43:17.240589 17407 consensus_queue.cc:1035] T 4a7aa3fe12e84eeca3f5297742f3f779 P fb063f3bf2c24ca29f5480118eaa718b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:43:17.241278 17377 tablet_replica.cc:439] T 4a7aa3fe12e84eeca3f5297742f3f779 P 1671965ee3334946868d67415747aba5: TxnStatusTablet state changed. Reason: New leader fb063f3bf2c24ca29f5480118eaa718b. Latest consensus state: current_term: 1 leader_uuid: "fb063f3bf2c24ca29f5480118eaa718b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } } }
I20260928 14:43:17.241382 17377 tablet_replica.cc:442] T 4a7aa3fe12e84eeca3f5297742f3f779 P 1671965ee3334946868d67415747aba5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:43:17.241333 17375 tablet_replica.cc:439] T 4a7aa3fe12e84eeca3f5297742f3f779 P 7f8b53fbbb804e189ce714679eb3e35c: TxnStatusTablet state changed. Reason: New leader fb063f3bf2c24ca29f5480118eaa718b. Latest consensus state: current_term: 1 leader_uuid: "fb063f3bf2c24ca29f5480118eaa718b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } } }
I20260928 14:43:17.241434 17375 tablet_replica.cc:442] T 4a7aa3fe12e84eeca3f5297742f3f779 P 7f8b53fbbb804e189ce714679eb3e35c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:43:17.242274 17374 tablet_replica.cc:439] T 4a7aa3fe12e84eeca3f5297742f3f779 P fb063f3bf2c24ca29f5480118eaa718b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fb063f3bf2c24ca29f5480118eaa718b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } } }
I20260928 14:43:17.242372 17374 tablet_replica.cc:442] T 4a7aa3fe12e84eeca3f5297742f3f779 P fb063f3bf2c24ca29f5480118eaa718b: This TxnStatusTablet replica's current role is: LEADER
I20260928 14:43:17.242421 17374 tablet_replica.cc:439] T 4a7aa3fe12e84eeca3f5297742f3f779 P fb063f3bf2c24ca29f5480118eaa718b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fb063f3bf2c24ca29f5480118eaa718b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } } }
I20260928 14:43:17.242466 17374 tablet_replica.cc:442] T 4a7aa3fe12e84eeca3f5297742f3f779 P fb063f3bf2c24ca29f5480118eaa718b: This TxnStatusTablet replica's current role is: LEADER
I20260928 14:43:17.242871 17375 tablet_replica.cc:439] T 4a7aa3fe12e84eeca3f5297742f3f779 P 7f8b53fbbb804e189ce714679eb3e35c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fb063f3bf2c24ca29f5480118eaa718b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } } }
I20260928 14:43:17.242972 17375 tablet_replica.cc:442] T 4a7aa3fe12e84eeca3f5297742f3f779 P 7f8b53fbbb804e189ce714679eb3e35c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:43:17.243218 17380 mvcc.cc:205] Tried to move back new op lower bound from 7334324622292905984 to 7334324622207963136. Current Snapshot: MvccSnapshot[applied={T|T < 7334324622292905984}]
I20260928 14:43:17.243325 17377 tablet_replica.cc:439] T 4a7aa3fe12e84eeca3f5297742f3f779 P 1671965ee3334946868d67415747aba5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fb063f3bf2c24ca29f5480118eaa718b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } } }
I20260928 14:43:17.243393 17377 tablet_replica.cc:442] T 4a7aa3fe12e84eeca3f5297742f3f779 P 1671965ee3334946868d67415747aba5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260928 14:43:17.255266 17416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:17.246698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:33699 (local address 127.9.33.67:38697)
0928 14:43:17.251964 (+ 5266us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:17.251982 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:17.252080 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:17.252344 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:17.252345 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:17.252347 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 14:43:17.252405 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:17.252406 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:17.253502 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:17.253503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:17.254201 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:17.254204 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:17.254722 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:17.254723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:17.254788 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:17.254814 (+ 26us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 14:43:17.255103 (+ 289us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 14:43:17.255105 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 14:43:17.255141 (+ 36us) client_negotiation.cc:240] Negotiation successful
0928 14:43:17.255189 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5185,"thread_start_us":80,"threads_started":1}
I20260928 14:43:17.255263 17366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:17.246828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:38697 (local address 127.9.33.65:33699)
0928 14:43:17.246910 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:17.246911 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:17.252123 (+ 5212us) server_negotiation.cc:407] Connection header received
0928 14:43:17.252182 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:17.252184 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:17.252204 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:17.252289 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:43:17.252560 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:17.253362 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:17.254312 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:17.254625 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:17.254703 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:17.254986 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:43:17.255048 (+ 62us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:43:17.255101 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:17.255149 (+ 48us) server_negotiation.cc:299] Negotiation successful
0928 14:43:17.255174 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 14:43:17.258129 17153 raft_consensus.cc:1261] T ca042f873a35462d98b61c5fefcc58d4 P 7f8b53fbbb804e189ce714679eb3e35c [term 1 FOLLOWER]: Refusing update from remote peer 1671965ee3334946868d67415747aba5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:43:17.258139 17013 raft_consensus.cc:1261] T ca042f873a35462d98b61c5fefcc58d4 P fb063f3bf2c24ca29f5480118eaa718b [term 1 FOLLOWER]: Refusing update from remote peer 1671965ee3334946868d67415747aba5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:43:17.258387 17377 consensus_queue.cc:1035] T ca042f873a35462d98b61c5fefcc58d4 P 1671965ee3334946868d67415747aba5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:43:17.258491 17373 consensus_queue.cc:1035] T ca042f873a35462d98b61c5fefcc58d4 P 1671965ee3334946868d67415747aba5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:43:17.268757 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:17.265545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:45245 (local address 127.9.33.67:35545)
0928 14:43:17.265590 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:17.265592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:17.265623 (+ 31us) server_negotiation.cc:407] Connection header received
0928 14:43:17.265749 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:17.265750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:17.265764 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:17.265826 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:43:17.265951 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:17.266752 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:17.267994 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:17.268203 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:17.268270 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:17.268494 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:43:17.268523 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:43:17.268568 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:17.268611 (+ 43us) server_negotiation.cc:299] Negotiation successful
0928 14:43:17.268688 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 14:43:17.268761 17421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:17.265307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:35545 (local address 127.9.33.65:45245)
0928 14:43:17.265551 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:17.265562 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:17.265644 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:17.265825 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:17.265826 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:17.265827 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 14:43:17.265894 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:17.265896 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:17.266852 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:17.266853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:17.267902 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:17.267906 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:17.268284 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:17.268285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:17.268367 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:17.268395 (+ 28us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 14:43:17.268564 (+ 169us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 14:43:17.268566 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 14:43:17.268609 (+ 43us) client_negotiation.cc:240] Negotiation successful
0928 14:43:17.268658 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":130,"threads_started":1}
I20260928 14:43:17.277312 16855 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:43:17.277446 16855 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:43:17.278172 16855 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0c41cc0a857d4203a8750d8a6b304591: Sending DeleteTablet for 3 replicas of tablet ca042f873a35462d98b61c5fefcc58d4
I20260928 14:43:17.278494 16993 tablet_service.cc:1514] Processing DeleteTablet for tablet ca042f873a35462d98b61c5fefcc58d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:43:17 UTC) from {username='oryx', principal='oryx/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:54694
I20260928 14:43:17.278508 17273 tablet_service.cc:1514] Processing DeleteTablet for tablet ca042f873a35462d98b61c5fefcc58d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:43:17 UTC) from {username='oryx', principal='oryx/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:36406
I20260928 14:43:17.278623 17391 tablet_replica.cc:330] T ca042f873a35462d98b61c5fefcc58d4 P fb063f3bf2c24ca29f5480118eaa718b: stopping tablet replica
I20260928 14:43:17.278651 17389 tablet_replica.cc:330] T ca042f873a35462d98b61c5fefcc58d4 P 1671965ee3334946868d67415747aba5: stopping tablet replica
I20260928 14:43:17.278734 17391 raft_consensus.cc:2229] T ca042f873a35462d98b61c5fefcc58d4 P fb063f3bf2c24ca29f5480118eaa718b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:43:17.278796 17391 raft_consensus.cc:2258] T ca042f873a35462d98b61c5fefcc58d4 P fb063f3bf2c24ca29f5480118eaa718b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:43:17.278811 17389 raft_consensus.cc:2229] T ca042f873a35462d98b61c5fefcc58d4 P 1671965ee3334946868d67415747aba5 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:43:17.278893 17389 raft_consensus.cc:2258] T ca042f873a35462d98b61c5fefcc58d4 P 1671965ee3334946868d67415747aba5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:43:17.279233 17391 ts_tablet_manager.cc:1905] T ca042f873a35462d98b61c5fefcc58d4 P fb063f3bf2c24ca29f5480118eaa718b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:43:17.280977 17389 ts_tablet_manager.cc:1905] T ca042f873a35462d98b61c5fefcc58d4 P 1671965ee3334946868d67415747aba5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:43:17.281026 17133 tablet_service.cc:1514] Processing DeleteTablet for tablet ca042f873a35462d98b61c5fefcc58d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:43:17 UTC) from {username='oryx', principal='oryx/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:39600
I20260928 14:43:17.281168 17390 tablet_replica.cc:330] T ca042f873a35462d98b61c5fefcc58d4 P 7f8b53fbbb804e189ce714679eb3e35c: stopping tablet replica
I20260928 14:43:17.281217 17390 raft_consensus.cc:2229] T ca042f873a35462d98b61c5fefcc58d4 P 7f8b53fbbb804e189ce714679eb3e35c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:43:17.281256 17390 raft_consensus.cc:2258] T ca042f873a35462d98b61c5fefcc58d4 P 7f8b53fbbb804e189ce714679eb3e35c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:43:17.281589 17390 ts_tablet_manager.cc:1905] T ca042f873a35462d98b61c5fefcc58d4 P 7f8b53fbbb804e189ce714679eb3e35c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:43:17.282971 17389 ts_tablet_manager.cc:1918] T ca042f873a35462d98b61c5fefcc58d4 P 1671965ee3334946868d67415747aba5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:43:17.283036 17389 log.cc:1196] T ca042f873a35462d98b61c5fefcc58d4 P 1671965ee3334946868d67415747aba5: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-2/wal/wals/ca042f873a35462d98b61c5fefcc58d4
I20260928 14:43:17.283211 17390 ts_tablet_manager.cc:1918] T ca042f873a35462d98b61c5fefcc58d4 P 7f8b53fbbb804e189ce714679eb3e35c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:43:17.283267 17390 log.cc:1196] T ca042f873a35462d98b61c5fefcc58d4 P 7f8b53fbbb804e189ce714679eb3e35c: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-1/wal/wals/ca042f873a35462d98b61c5fefcc58d4
I20260928 14:43:17.283272 17389 ts_tablet_manager.cc:1939] T ca042f873a35462d98b61c5fefcc58d4 P 1671965ee3334946868d67415747aba5: Deleting consensus metadata
I20260928 14:43:17.283511 17390 ts_tablet_manager.cc:1939] T ca042f873a35462d98b61c5fefcc58d4 P 7f8b53fbbb804e189ce714679eb3e35c: Deleting consensus metadata
I20260928 14:43:17.283784 16840 catalog_manager.cc:4841] TS 1671965ee3334946868d67415747aba5 (127.9.33.67:35545): tablet ca042f873a35462d98b61c5fefcc58d4 (table test-table [id=24f6c19f1a8942b089260ee1fdfb8ce5]) successfully deleted
I20260928 14:43:17.284253 16841 catalog_manager.cc:4841] TS 7f8b53fbbb804e189ce714679eb3e35c (127.9.33.66:44523): tablet ca042f873a35462d98b61c5fefcc58d4 (table test-table [id=24f6c19f1a8942b089260ee1fdfb8ce5]) successfully deleted
I20260928 14:43:17.284369 17391 ts_tablet_manager.cc:1918] T ca042f873a35462d98b61c5fefcc58d4 P fb063f3bf2c24ca29f5480118eaa718b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:43:17.284426 17391 log.cc:1196] T ca042f873a35462d98b61c5fefcc58d4 P fb063f3bf2c24ca29f5480118eaa718b: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-0/wal/wals/ca042f873a35462d98b61c5fefcc58d4
I20260928 14:43:17.284598 17391 ts_tablet_manager.cc:1939] T ca042f873a35462d98b61c5fefcc58d4 P fb063f3bf2c24ca29f5480118eaa718b: Deleting consensus metadata
I20260928 14:43:17.284925 16842 catalog_manager.cc:4841] TS fb063f3bf2c24ca29f5480118eaa718b (127.9.33.65:33699): tablet ca042f873a35462d98b61c5fefcc58d4 (table test-table [id=24f6c19f1a8942b089260ee1fdfb8ce5]) successfully deleted
Sep 28 14:43:17 dist-test-slave-pv0w krb5kdc[16811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 28 14:43:17 dist-test-slave-pv0w krb5kdc[16811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:17 dist-test-slave-pv0w krb5kdc[16811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
W20260928 14:43:17.308110 16905 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:17.301005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46716 (local address 127.9.33.126:36113)
0928 14:43:17.301070 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:17.301073 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:17.301175 (+ 102us) server_negotiation.cc:407] Connection header received
0928 14:43:17.301324 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:17.301325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:17.301337 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:17.301409 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:17.302212 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:17.302793 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:17.303914 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:17.304156 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:17.304284 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:17.308021 (+ 3737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46716: BlockingRecv error: recv got EOF from 127.0.0.1:46716 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 28 14:43:17 dist-test-slave-pv0w krb5kdc[16811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM
I20260928 14:43:17.318712 16905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:17.309250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46732 (local address 127.9.33.126:36113)
0928 14:43:17.309317 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:17.309319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:17.309459 (+ 140us) server_negotiation.cc:407] Connection header received
0928 14:43:17.309500 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:17.309501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:17.309514 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:17.309590 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:17.310121 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:17.310918 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:17.311808 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:17.312212 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:17.312329 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:17.315238 (+ 2909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:17.315241 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:17.315241 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:17.315252 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:17.317324 (+ 2072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:17.317751 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:17.317753 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:17.317753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:17.317824 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:17.318102 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:17.318104 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:17.318104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:17.318261 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:17.318383 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:17.318519 (+ 136us) server_negotiation.cc:299] Negotiation successful
0928 14:43:17.318610 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260928 14:43:17.320911 16855 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46732:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:43:17.321137 16855 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:43:17.322856 16993 tablet_service.cc:1467] Processing CreateTablet for tablet c06b3f608e3a48ef8dd8b8276bab03b6 (DEFAULT_TABLE table=test-table [id=44ca0c8d84d14c3594ffc288dba178f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:43:17.322891 17133 tablet_service.cc:1467] Processing CreateTablet for tablet c06b3f608e3a48ef8dd8b8276bab03b6 (DEFAULT_TABLE table=test-table [id=44ca0c8d84d14c3594ffc288dba178f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:43:17.323033 16993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c06b3f608e3a48ef8dd8b8276bab03b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:17.323033 17133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c06b3f608e3a48ef8dd8b8276bab03b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:17.324702 17368 tablet_bootstrap.cc:492] T c06b3f608e3a48ef8dd8b8276bab03b6 P fb063f3bf2c24ca29f5480118eaa718b: Bootstrap starting.
I20260928 14:43:17.324877 17273 tablet_service.cc:1467] Processing CreateTablet for tablet c06b3f608e3a48ef8dd8b8276bab03b6 (DEFAULT_TABLE table=test-table [id=44ca0c8d84d14c3594ffc288dba178f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:43:17.325018 17273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c06b3f608e3a48ef8dd8b8276bab03b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:17.325181 17368 tablet_bootstrap.cc:654] T c06b3f608e3a48ef8dd8b8276bab03b6 P fb063f3bf2c24ca29f5480118eaa718b: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:17.325809 17368 tablet_bootstrap.cc:492] T c06b3f608e3a48ef8dd8b8276bab03b6 P fb063f3bf2c24ca29f5480118eaa718b: No bootstrap required, opened a new log
I20260928 14:43:17.325881 17368 ts_tablet_manager.cc:1397] T c06b3f608e3a48ef8dd8b8276bab03b6 P fb063f3bf2c24ca29f5480118eaa718b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260928 14:43:17.326076 17368 raft_consensus.cc:348] T c06b3f608e3a48ef8dd8b8276bab03b6 P fb063f3bf2c24ca29f5480118eaa718b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } }
I20260928 14:43:17.326165 17368 raft_consensus.cc:374] T c06b3f608e3a48ef8dd8b8276bab03b6 P fb063f3bf2c24ca29f5480118eaa718b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:17.326195 17368 raft_consensus.cc:729] T c06b3f608e3a48ef8dd8b8276bab03b6 P fb063f3bf2c24ca29f5480118eaa718b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb063f3bf2c24ca29f5480118eaa718b, State: Initialized, Role: FOLLOWER
I20260928 14:43:17.326237 17368 consensus_queue.cc:260] T c06b3f608e3a48ef8dd8b8276bab03b6 P fb063f3bf2c24ca29f5480118eaa718b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } }
I20260928 14:43:17.326337 17368 ts_tablet_manager.cc:1428] T c06b3f608e3a48ef8dd8b8276bab03b6 P fb063f3bf2c24ca29f5480118eaa718b: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260928 14:43:17.326454 17371 tablet_bootstrap.cc:492] T c06b3f608e3a48ef8dd8b8276bab03b6 P 7f8b53fbbb804e189ce714679eb3e35c: Bootstrap starting.
I20260928 14:43:17.326937 17371 tablet_bootstrap.cc:654] T c06b3f608e3a48ef8dd8b8276bab03b6 P 7f8b53fbbb804e189ce714679eb3e35c: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:17.327355 17367 tablet_bootstrap.cc:492] T c06b3f608e3a48ef8dd8b8276bab03b6 P 1671965ee3334946868d67415747aba5: Bootstrap starting.
I20260928 14:43:17.327739 17367 tablet_bootstrap.cc:654] T c06b3f608e3a48ef8dd8b8276bab03b6 P 1671965ee3334946868d67415747aba5: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:17.327766 17371 tablet_bootstrap.cc:492] T c06b3f608e3a48ef8dd8b8276bab03b6 P 7f8b53fbbb804e189ce714679eb3e35c: No bootstrap required, opened a new log
I20260928 14:43:17.327827 17371 ts_tablet_manager.cc:1397] T c06b3f608e3a48ef8dd8b8276bab03b6 P 7f8b53fbbb804e189ce714679eb3e35c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260928 14:43:17.327996 17371 raft_consensus.cc:348] T c06b3f608e3a48ef8dd8b8276bab03b6 P 7f8b53fbbb804e189ce714679eb3e35c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } }
I20260928 14:43:17.328114 17371 raft_consensus.cc:374] T c06b3f608e3a48ef8dd8b8276bab03b6 P 7f8b53fbbb804e189ce714679eb3e35c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:17.328143 17371 raft_consensus.cc:729] T c06b3f608e3a48ef8dd8b8276bab03b6 P 7f8b53fbbb804e189ce714679eb3e35c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f8b53fbbb804e189ce714679eb3e35c, State: Initialized, Role: FOLLOWER
I20260928 14:43:17.328189 17371 consensus_queue.cc:260] T c06b3f608e3a48ef8dd8b8276bab03b6 P 7f8b53fbbb804e189ce714679eb3e35c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } }
I20260928 14:43:17.328302 17371 ts_tablet_manager.cc:1428] T c06b3f608e3a48ef8dd8b8276bab03b6 P 7f8b53fbbb804e189ce714679eb3e35c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:17.328589 17367 tablet_bootstrap.cc:492] T c06b3f608e3a48ef8dd8b8276bab03b6 P 1671965ee3334946868d67415747aba5: No bootstrap required, opened a new log
I20260928 14:43:17.328645 17367 ts_tablet_manager.cc:1397] T c06b3f608e3a48ef8dd8b8276bab03b6 P 1671965ee3334946868d67415747aba5: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260928 14:43:17.328763 17367 raft_consensus.cc:348] T c06b3f608e3a48ef8dd8b8276bab03b6 P 1671965ee3334946868d67415747aba5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } }
I20260928 14:43:17.328830 17367 raft_consensus.cc:374] T c06b3f608e3a48ef8dd8b8276bab03b6 P 1671965ee3334946868d67415747aba5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:17.328930 17367 raft_consensus.cc:729] T c06b3f608e3a48ef8dd8b8276bab03b6 P 1671965ee3334946868d67415747aba5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1671965ee3334946868d67415747aba5, State: Initialized, Role: FOLLOWER
I20260928 14:43:17.328982 17367 consensus_queue.cc:260] T c06b3f608e3a48ef8dd8b8276bab03b6 P 1671965ee3334946868d67415747aba5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } }
I20260928 14:43:17.329084 17367 ts_tablet_manager.cc:1428] T c06b3f608e3a48ef8dd8b8276bab03b6 P 1671965ee3334946868d67415747aba5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:17.720257 17417 raft_consensus.cc:482] T c06b3f608e3a48ef8dd8b8276bab03b6 P fb063f3bf2c24ca29f5480118eaa718b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:43:17.720383 17417 raft_consensus.cc:504] T c06b3f608e3a48ef8dd8b8276bab03b6 P fb063f3bf2c24ca29f5480118eaa718b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } }
I20260928 14:43:17.720563 17417 leader_election.cc:288] T c06b3f608e3a48ef8dd8b8276bab03b6 P fb063f3bf2c24ca29f5480118eaa718b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f8b53fbbb804e189ce714679eb3e35c (127.9.33.66:44523), 1671965ee3334946868d67415747aba5 (127.9.33.67:35545)
I20260928 14:43:17.720814 17153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c06b3f608e3a48ef8dd8b8276bab03b6" candidate_uuid: "fb063f3bf2c24ca29f5480118eaa718b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f8b53fbbb804e189ce714679eb3e35c" is_pre_election: true
I20260928 14:43:17.720918 17153 raft_consensus.cc:2454] T c06b3f608e3a48ef8dd8b8276bab03b6 P 7f8b53fbbb804e189ce714679eb3e35c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb063f3bf2c24ca29f5480118eaa718b in term 0.
I20260928 14:43:17.720860 17293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c06b3f608e3a48ef8dd8b8276bab03b6" candidate_uuid: "fb063f3bf2c24ca29f5480118eaa718b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1671965ee3334946868d67415747aba5" is_pre_election: true
I20260928 14:43:17.721027 17293 raft_consensus.cc:2454] T c06b3f608e3a48ef8dd8b8276bab03b6 P 1671965ee3334946868d67415747aba5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb063f3bf2c24ca29f5480118eaa718b in term 0.
I20260928 14:43:17.721112 16945 leader_election.cc:302] T c06b3f608e3a48ef8dd8b8276bab03b6 P fb063f3bf2c24ca29f5480118eaa718b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f8b53fbbb804e189ce714679eb3e35c, fb063f3bf2c24ca29f5480118eaa718b; no voters:
I20260928 14:43:17.721220 17417 raft_consensus.cc:2781] T c06b3f608e3a48ef8dd8b8276bab03b6 P fb063f3bf2c24ca29f5480118eaa718b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:43:17.721272 17417 raft_consensus.cc:482] T c06b3f608e3a48ef8dd8b8276bab03b6 P fb063f3bf2c24ca29f5480118eaa718b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:43:17.721309 17417 raft_consensus.cc:3037] T c06b3f608e3a48ef8dd8b8276bab03b6 P fb063f3bf2c24ca29f5480118eaa718b [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:17.722075 17417 raft_consensus.cc:504] T c06b3f608e3a48ef8dd8b8276bab03b6 P fb063f3bf2c24ca29f5480118eaa718b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } }
I20260928 14:43:17.722225 17417 leader_election.cc:288] T c06b3f608e3a48ef8dd8b8276bab03b6 P fb063f3bf2c24ca29f5480118eaa718b [CANDIDATE]: Term 1 election: Requested vote from peers 7f8b53fbbb804e189ce714679eb3e35c (127.9.33.66:44523), 1671965ee3334946868d67415747aba5 (127.9.33.67:35545)
I20260928 14:43:17.722416 17153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c06b3f608e3a48ef8dd8b8276bab03b6" candidate_uuid: "fb063f3bf2c24ca29f5480118eaa718b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f8b53fbbb804e189ce714679eb3e35c"
I20260928 14:43:17.722435 17293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c06b3f608e3a48ef8dd8b8276bab03b6" candidate_uuid: "fb063f3bf2c24ca29f5480118eaa718b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1671965ee3334946868d67415747aba5"
I20260928 14:43:17.722510 17293 raft_consensus.cc:3037] T c06b3f608e3a48ef8dd8b8276bab03b6 P 1671965ee3334946868d67415747aba5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:17.722510 17153 raft_consensus.cc:3037] T c06b3f608e3a48ef8dd8b8276bab03b6 P 7f8b53fbbb804e189ce714679eb3e35c [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:17.723299 17153 raft_consensus.cc:2454] T c06b3f608e3a48ef8dd8b8276bab03b6 P 7f8b53fbbb804e189ce714679eb3e35c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb063f3bf2c24ca29f5480118eaa718b in term 1.
I20260928 14:43:17.723299 17293 raft_consensus.cc:2454] T c06b3f608e3a48ef8dd8b8276bab03b6 P 1671965ee3334946868d67415747aba5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb063f3bf2c24ca29f5480118eaa718b in term 1.
I20260928 14:43:17.723518 16945 leader_election.cc:302] T c06b3f608e3a48ef8dd8b8276bab03b6 P fb063f3bf2c24ca29f5480118eaa718b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f8b53fbbb804e189ce714679eb3e35c, fb063f3bf2c24ca29f5480118eaa718b; no voters:
I20260928 14:43:17.723654 17417 raft_consensus.cc:2781] T c06b3f608e3a48ef8dd8b8276bab03b6 P fb063f3bf2c24ca29f5480118eaa718b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:43:17.723742 17417 raft_consensus.cc:686] T c06b3f608e3a48ef8dd8b8276bab03b6 P fb063f3bf2c24ca29f5480118eaa718b [term 1 LEADER]: Becoming Leader. State: Replica: fb063f3bf2c24ca29f5480118eaa718b, State: Running, Role: LEADER
I20260928 14:43:17.723802 17417 consensus_queue.cc:237] T c06b3f608e3a48ef8dd8b8276bab03b6 P fb063f3bf2c24ca29f5480118eaa718b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } }
I20260928 14:43:17.724568 16855 catalog_manager.cc:5495] T c06b3f608e3a48ef8dd8b8276bab03b6 P fb063f3bf2c24ca29f5480118eaa718b reported cstate change: term changed from 0 to 1, leader changed from <none> to fb063f3bf2c24ca29f5480118eaa718b (127.9.33.65). New cstate: current_term: 1 leader_uuid: "fb063f3bf2c24ca29f5480118eaa718b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } health_report { overall_health: UNKNOWN } } }
I20260928 14:43:17.779295 17441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:17.776540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54724 (local address 127.9.33.65:33699)
0928 14:43:17.776655 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:17.776657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:17.776687 (+ 30us) server_negotiation.cc:407] Connection header received
0928 14:43:17.776753 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:17.776753 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:17.776765 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:17.776827 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:43:17.777060 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:17.777589 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:17.778548 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:17.778753 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:17.778810 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:17.778977 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:43:17.779006 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:43:17.779043 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:17.779158 (+ 115us) server_negotiation.cc:299] Negotiation successful
0928 14:43:17.779179 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":55,"threads_started":1}
I20260928 14:43:17.780400 17153 raft_consensus.cc:1261] T c06b3f608e3a48ef8dd8b8276bab03b6 P 7f8b53fbbb804e189ce714679eb3e35c [term 1 FOLLOWER]: Refusing update from remote peer fb063f3bf2c24ca29f5480118eaa718b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:43:17.780400 17293 raft_consensus.cc:1261] T c06b3f608e3a48ef8dd8b8276bab03b6 P 1671965ee3334946868d67415747aba5 [term 1 FOLLOWER]: Refusing update from remote peer fb063f3bf2c24ca29f5480118eaa718b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:43:17.780732 17417 consensus_queue.cc:1035] T c06b3f608e3a48ef8dd8b8276bab03b6 P fb063f3bf2c24ca29f5480118eaa718b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:43:17.780932 17374 consensus_queue.cc:1035] T c06b3f608e3a48ef8dd8b8276bab03b6 P fb063f3bf2c24ca29f5480118eaa718b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:43:17.784369 16856 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:43:17.784471 16856 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:43:17.785110 16856 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0c41cc0a857d4203a8750d8a6b304591: Sending DeleteTablet for 3 replicas of tablet c06b3f608e3a48ef8dd8b8276bab03b6
I20260928 14:43:17.785403 16993 tablet_service.cc:1514] Processing DeleteTablet for tablet c06b3f608e3a48ef8dd8b8276bab03b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:43:17 UTC) from {username='oryx', principal='oryx/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:54694
I20260928 14:43:17.785421 17273 tablet_service.cc:1514] Processing DeleteTablet for tablet c06b3f608e3a48ef8dd8b8276bab03b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:43:17 UTC) from {username='oryx', principal='oryx/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:36406
I20260928 14:43:17.785472 17133 tablet_service.cc:1514] Processing DeleteTablet for tablet c06b3f608e3a48ef8dd8b8276bab03b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:43:17 UTC) from {username='oryx', principal='oryx/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:39600
I20260928 14:43:17.785562 17452 tablet_replica.cc:330] T c06b3f608e3a48ef8dd8b8276bab03b6 P 1671965ee3334946868d67415747aba5: stopping tablet replica
I20260928 14:43:17.785627 17452 raft_consensus.cc:2229] T c06b3f608e3a48ef8dd8b8276bab03b6 P 1671965ee3334946868d67415747aba5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:43:17.785671 17452 raft_consensus.cc:2258] T c06b3f608e3a48ef8dd8b8276bab03b6 P 1671965ee3334946868d67415747aba5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:43:17.785667 16856 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46732:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:43:17.785817 16856 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:43:17.785831 17451 tablet_replica.cc:330] T c06b3f608e3a48ef8dd8b8276bab03b6 P fb063f3bf2c24ca29f5480118eaa718b: stopping tablet replica
I20260928 14:43:17.785881 17451 raft_consensus.cc:2229] T c06b3f608e3a48ef8dd8b8276bab03b6 P fb063f3bf2c24ca29f5480118eaa718b [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:43:17.785974 17451 raft_consensus.cc:2258] T c06b3f608e3a48ef8dd8b8276bab03b6 P fb063f3bf2c24ca29f5480118eaa718b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:43:17.785983 17453 tablet_replica.cc:330] T c06b3f608e3a48ef8dd8b8276bab03b6 P 7f8b53fbbb804e189ce714679eb3e35c: stopping tablet replica
I20260928 14:43:17.786124 17453 raft_consensus.cc:2229] T c06b3f608e3a48ef8dd8b8276bab03b6 P 7f8b53fbbb804e189ce714679eb3e35c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:43:17.786265 17453 raft_consensus.cc:2258] T c06b3f608e3a48ef8dd8b8276bab03b6 P 7f8b53fbbb804e189ce714679eb3e35c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:43:17.786654 17452 ts_tablet_manager.cc:1905] T c06b3f608e3a48ef8dd8b8276bab03b6 P 1671965ee3334946868d67415747aba5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:43:17.787158 17133 tablet_service.cc:1467] Processing CreateTablet for tablet d97d6030164e453aa0f6440e85426695 (DEFAULT_TABLE table=test-table [id=154e79b59b0c4603a4d3145a169e710c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:43:17.787205 17273 tablet_service.cc:1467] Processing CreateTablet for tablet d97d6030164e453aa0f6440e85426695 (DEFAULT_TABLE table=test-table [id=154e79b59b0c4603a4d3145a169e710c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:43:17.787282 17133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d97d6030164e453aa0f6440e85426695. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:17.787313 17273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d97d6030164e453aa0f6440e85426695. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:17.788507 17367 tablet_bootstrap.cc:492] T d97d6030164e453aa0f6440e85426695 P 1671965ee3334946868d67415747aba5: Bootstrap starting.
I20260928 14:43:17.788800 17371 tablet_bootstrap.cc:492] T d97d6030164e453aa0f6440e85426695 P 7f8b53fbbb804e189ce714679eb3e35c: Bootstrap starting.
I20260928 14:43:17.788898 17367 tablet_bootstrap.cc:654] T d97d6030164e453aa0f6440e85426695 P 1671965ee3334946868d67415747aba5: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:17.789160 17371 tablet_bootstrap.cc:654] T d97d6030164e453aa0f6440e85426695 P 7f8b53fbbb804e189ce714679eb3e35c: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:17.789161 16993 tablet_service.cc:1467] Processing CreateTablet for tablet d97d6030164e453aa0f6440e85426695 (DEFAULT_TABLE table=test-table [id=154e79b59b0c4603a4d3145a169e710c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:43:17.789266 16993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d97d6030164e453aa0f6440e85426695. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:17.789762 17371 tablet_bootstrap.cc:492] T d97d6030164e453aa0f6440e85426695 P 7f8b53fbbb804e189ce714679eb3e35c: No bootstrap required, opened a new log
I20260928 14:43:17.789841 17371 ts_tablet_manager.cc:1397] T d97d6030164e453aa0f6440e85426695 P 7f8b53fbbb804e189ce714679eb3e35c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:17.790004 17371 raft_consensus.cc:348] T d97d6030164e453aa0f6440e85426695 P 7f8b53fbbb804e189ce714679eb3e35c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } }
I20260928 14:43:17.790074 17371 raft_consensus.cc:374] T d97d6030164e453aa0f6440e85426695 P 7f8b53fbbb804e189ce714679eb3e35c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:17.790100 17371 raft_consensus.cc:729] T d97d6030164e453aa0f6440e85426695 P 7f8b53fbbb804e189ce714679eb3e35c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f8b53fbbb804e189ce714679eb3e35c, State: Initialized, Role: FOLLOWER
I20260928 14:43:17.790132 17371 consensus_queue.cc:260] T d97d6030164e453aa0f6440e85426695 P 7f8b53fbbb804e189ce714679eb3e35c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } }
I20260928 14:43:17.790218 17371 ts_tablet_manager.cc:1428] T d97d6030164e453aa0f6440e85426695 P 7f8b53fbbb804e189ce714679eb3e35c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:17.790727 17368 tablet_bootstrap.cc:492] T d97d6030164e453aa0f6440e85426695 P fb063f3bf2c24ca29f5480118eaa718b: Bootstrap starting.
I20260928 14:43:17.791069 17368 tablet_bootstrap.cc:654] T d97d6030164e453aa0f6440e85426695 P fb063f3bf2c24ca29f5480118eaa718b: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:17.791453 17367 tablet_bootstrap.cc:492] T d97d6030164e453aa0f6440e85426695 P 1671965ee3334946868d67415747aba5: No bootstrap required, opened a new log
I20260928 14:43:17.791512 17367 ts_tablet_manager.cc:1397] T d97d6030164e453aa0f6440e85426695 P 1671965ee3334946868d67415747aba5: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260928 14:43:17.791633 17367 raft_consensus.cc:348] T d97d6030164e453aa0f6440e85426695 P 1671965ee3334946868d67415747aba5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } }
I20260928 14:43:17.791743 17367 raft_consensus.cc:374] T d97d6030164e453aa0f6440e85426695 P 1671965ee3334946868d67415747aba5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:17.791769 17367 raft_consensus.cc:729] T d97d6030164e453aa0f6440e85426695 P 1671965ee3334946868d67415747aba5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1671965ee3334946868d67415747aba5, State: Initialized, Role: FOLLOWER
I20260928 14:43:17.791803 17368 tablet_bootstrap.cc:492] T d97d6030164e453aa0f6440e85426695 P fb063f3bf2c24ca29f5480118eaa718b: No bootstrap required, opened a new log
I20260928 14:43:17.791841 17368 ts_tablet_manager.cc:1397] T d97d6030164e453aa0f6440e85426695 P fb063f3bf2c24ca29f5480118eaa718b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260928 14:43:17.791822 17367 consensus_queue.cc:260] T d97d6030164e453aa0f6440e85426695 P 1671965ee3334946868d67415747aba5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } }
I20260928 14:43:17.791937 17367 ts_tablet_manager.cc:1428] T d97d6030164e453aa0f6440e85426695 P 1671965ee3334946868d67415747aba5: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:17.791958 17368 raft_consensus.cc:348] T d97d6030164e453aa0f6440e85426695 P fb063f3bf2c24ca29f5480118eaa718b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } }
I20260928 14:43:17.792026 17368 raft_consensus.cc:374] T d97d6030164e453aa0f6440e85426695 P fb063f3bf2c24ca29f5480118eaa718b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:17.792052 17368 raft_consensus.cc:729] T d97d6030164e453aa0f6440e85426695 P fb063f3bf2c24ca29f5480118eaa718b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb063f3bf2c24ca29f5480118eaa718b, State: Initialized, Role: FOLLOWER
I20260928 14:43:17.792093 17368 consensus_queue.cc:260] T d97d6030164e453aa0f6440e85426695 P fb063f3bf2c24ca29f5480118eaa718b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } }
I20260928 14:43:17.792120 17453 ts_tablet_manager.cc:1905] T c06b3f608e3a48ef8dd8b8276bab03b6 P 7f8b53fbbb804e189ce714679eb3e35c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:43:17.792172 17368 ts_tablet_manager.cc:1428] T d97d6030164e453aa0f6440e85426695 P fb063f3bf2c24ca29f5480118eaa718b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:17.792764 17451 ts_tablet_manager.cc:1905] T c06b3f608e3a48ef8dd8b8276bab03b6 P fb063f3bf2c24ca29f5480118eaa718b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:43:17.794201 17451 ts_tablet_manager.cc:1918] T c06b3f608e3a48ef8dd8b8276bab03b6 P fb063f3bf2c24ca29f5480118eaa718b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:43:17.794263 17451 log.cc:1196] T c06b3f608e3a48ef8dd8b8276bab03b6 P fb063f3bf2c24ca29f5480118eaa718b: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-0/wal/wals/c06b3f608e3a48ef8dd8b8276bab03b6
I20260928 14:43:17.794492 17451 ts_tablet_manager.cc:1939] T c06b3f608e3a48ef8dd8b8276bab03b6 P fb063f3bf2c24ca29f5480118eaa718b: Deleting consensus metadata
I20260928 14:43:17.794836 16842 catalog_manager.cc:4841] TS fb063f3bf2c24ca29f5480118eaa718b (127.9.33.65:33699): tablet c06b3f608e3a48ef8dd8b8276bab03b6 (table test-table [id=44ca0c8d84d14c3594ffc288dba178f5]) successfully deleted
I20260928 14:43:17.794840 17453 ts_tablet_manager.cc:1918] T c06b3f608e3a48ef8dd8b8276bab03b6 P 7f8b53fbbb804e189ce714679eb3e35c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:43:17.794907 17453 log.cc:1196] T c06b3f608e3a48ef8dd8b8276bab03b6 P 7f8b53fbbb804e189ce714679eb3e35c: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-1/wal/wals/c06b3f608e3a48ef8dd8b8276bab03b6
I20260928 14:43:17.795136 17453 ts_tablet_manager.cc:1939] T c06b3f608e3a48ef8dd8b8276bab03b6 P 7f8b53fbbb804e189ce714679eb3e35c: Deleting consensus metadata
I20260928 14:43:17.795459 16841 catalog_manager.cc:4841] TS 7f8b53fbbb804e189ce714679eb3e35c (127.9.33.66:44523): tablet c06b3f608e3a48ef8dd8b8276bab03b6 (table test-table [id=44ca0c8d84d14c3594ffc288dba178f5]) successfully deleted
I20260928 14:43:17.795854 17452 ts_tablet_manager.cc:1918] T c06b3f608e3a48ef8dd8b8276bab03b6 P 1671965ee3334946868d67415747aba5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:43:17.795909 17452 log.cc:1196] T c06b3f608e3a48ef8dd8b8276bab03b6 P 1671965ee3334946868d67415747aba5: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-2/wal/wals/c06b3f608e3a48ef8dd8b8276bab03b6
I20260928 14:43:17.796113 17452 ts_tablet_manager.cc:1939] T c06b3f608e3a48ef8dd8b8276bab03b6 P 1671965ee3334946868d67415747aba5: Deleting consensus metadata
I20260928 14:43:17.796403 16840 catalog_manager.cc:4841] TS 1671965ee3334946868d67415747aba5 (127.9.33.67:35545): tablet c06b3f608e3a48ef8dd8b8276bab03b6 (table test-table [id=44ca0c8d84d14c3594ffc288dba178f5]) successfully deleted
I20260928 14:43:17.952142 17417 raft_consensus.cc:482] T d97d6030164e453aa0f6440e85426695 P fb063f3bf2c24ca29f5480118eaa718b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:43:17.952258 17417 raft_consensus.cc:504] T d97d6030164e453aa0f6440e85426695 P fb063f3bf2c24ca29f5480118eaa718b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } }
I20260928 14:43:17.952454 17417 leader_election.cc:288] T d97d6030164e453aa0f6440e85426695 P fb063f3bf2c24ca29f5480118eaa718b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f8b53fbbb804e189ce714679eb3e35c (127.9.33.66:44523), 1671965ee3334946868d67415747aba5 (127.9.33.67:35545)
I20260928 14:43:17.952705 17153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d97d6030164e453aa0f6440e85426695" candidate_uuid: "fb063f3bf2c24ca29f5480118eaa718b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f8b53fbbb804e189ce714679eb3e35c" is_pre_election: true
I20260928 14:43:17.952744 17293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d97d6030164e453aa0f6440e85426695" candidate_uuid: "fb063f3bf2c24ca29f5480118eaa718b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1671965ee3334946868d67415747aba5" is_pre_election: true
I20260928 14:43:17.952822 17293 raft_consensus.cc:2454] T d97d6030164e453aa0f6440e85426695 P 1671965ee3334946868d67415747aba5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb063f3bf2c24ca29f5480118eaa718b in term 0.
I20260928 14:43:17.952843 17153 raft_consensus.cc:2454] T d97d6030164e453aa0f6440e85426695 P 7f8b53fbbb804e189ce714679eb3e35c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb063f3bf2c24ca29f5480118eaa718b in term 0.
I20260928 14:43:17.953006 16944 leader_election.cc:302] T d97d6030164e453aa0f6440e85426695 P fb063f3bf2c24ca29f5480118eaa718b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1671965ee3334946868d67415747aba5, fb063f3bf2c24ca29f5480118eaa718b; no voters:
I20260928 14:43:17.953158 17417 raft_consensus.cc:2781] T d97d6030164e453aa0f6440e85426695 P fb063f3bf2c24ca29f5480118eaa718b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:43:17.953202 17417 raft_consensus.cc:482] T d97d6030164e453aa0f6440e85426695 P fb063f3bf2c24ca29f5480118eaa718b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:43:17.953215 17417 raft_consensus.cc:3037] T d97d6030164e453aa0f6440e85426695 P fb063f3bf2c24ca29f5480118eaa718b [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:17.953896 17417 raft_consensus.cc:504] T d97d6030164e453aa0f6440e85426695 P fb063f3bf2c24ca29f5480118eaa718b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } }
I20260928 14:43:17.954087 17417 leader_election.cc:288] T d97d6030164e453aa0f6440e85426695 P fb063f3bf2c24ca29f5480118eaa718b [CANDIDATE]: Term 1 election: Requested vote from peers 7f8b53fbbb804e189ce714679eb3e35c (127.9.33.66:44523), 1671965ee3334946868d67415747aba5 (127.9.33.67:35545)
I20260928 14:43:17.954252 17153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d97d6030164e453aa0f6440e85426695" candidate_uuid: "fb063f3bf2c24ca29f5480118eaa718b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f8b53fbbb804e189ce714679eb3e35c"
I20260928 14:43:17.954262 17293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d97d6030164e453aa0f6440e85426695" candidate_uuid: "fb063f3bf2c24ca29f5480118eaa718b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1671965ee3334946868d67415747aba5"
I20260928 14:43:17.954352 17153 raft_consensus.cc:3037] T d97d6030164e453aa0f6440e85426695 P 7f8b53fbbb804e189ce714679eb3e35c [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:17.954373 17293 raft_consensus.cc:3037] T d97d6030164e453aa0f6440e85426695 P 1671965ee3334946868d67415747aba5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:17.954999 17293 raft_consensus.cc:2454] T d97d6030164e453aa0f6440e85426695 P 1671965ee3334946868d67415747aba5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb063f3bf2c24ca29f5480118eaa718b in term 1.
I20260928 14:43:17.954999 17153 raft_consensus.cc:2454] T d97d6030164e453aa0f6440e85426695 P 7f8b53fbbb804e189ce714679eb3e35c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb063f3bf2c24ca29f5480118eaa718b in term 1.
I20260928 14:43:17.955233 16945 leader_election.cc:302] T d97d6030164e453aa0f6440e85426695 P fb063f3bf2c24ca29f5480118eaa718b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1671965ee3334946868d67415747aba5, 7f8b53fbbb804e189ce714679eb3e35c, fb063f3bf2c24ca29f5480118eaa718b; no voters:
I20260928 14:43:17.955343 17417 raft_consensus.cc:2781] T d97d6030164e453aa0f6440e85426695 P fb063f3bf2c24ca29f5480118eaa718b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:43:17.955416 17417 raft_consensus.cc:686] T d97d6030164e453aa0f6440e85426695 P fb063f3bf2c24ca29f5480118eaa718b [term 1 LEADER]: Becoming Leader. State: Replica: fb063f3bf2c24ca29f5480118eaa718b, State: Running, Role: LEADER
I20260928 14:43:17.955472 17417 consensus_queue.cc:237] T d97d6030164e453aa0f6440e85426695 P fb063f3bf2c24ca29f5480118eaa718b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } }
I20260928 14:43:17.956139 16855 catalog_manager.cc:5495] T d97d6030164e453aa0f6440e85426695 P fb063f3bf2c24ca29f5480118eaa718b reported cstate change: term changed from 0 to 1, leader changed from <none> to fb063f3bf2c24ca29f5480118eaa718b (127.9.33.65). New cstate: current_term: 1 leader_uuid: "fb063f3bf2c24ca29f5480118eaa718b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb063f3bf2c24ca29f5480118eaa718b" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 } health_report { overall_health: UNKNOWN } } }
I20260928 14:43:17.982959 17441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:17.980767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:54451 (local address 127.9.33.65:33699)
0928 14:43:17.980815 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:17.980817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:17.980942 (+ 125us) server_negotiation.cc:407] Connection header received
0928 14:43:17.980993 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:17.980994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:17.981003 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:17.981048 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:43:17.981135 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:17.981624 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:17.982371 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:17.982555 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:17.982612 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:17.982740 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:43:17.982777 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:43:17.982816 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:17.982872 (+ 56us) server_negotiation.cc:299] Negotiation successful
0928 14:43:17.982893 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":25}
I20260928 14:43:17.982965 17461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:17.980685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:33699 (local address 127.9.33.65:54451)
0928 14:43:17.980889 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:17.980897 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:17.980947 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:17.981051 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:17.981052 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:17.981053 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0928 14:43:17.981099 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:17.981100 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:17.981697 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:17.981698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:17.982275 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:17.982277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:17.982631 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:17.982631 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:17.982671 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:17.982675 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0928 14:43:17.982834 (+ 159us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0928 14:43:17.982835 (+ 1us) client_negotiation.cc:769] Sending connection context
0928 14:43:17.982869 (+ 34us) client_negotiation.cc:240] Negotiation successful
0928 14:43:17.982905 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":170,"threads_started":1}
I20260928 14:43:17.985028 17153 raft_consensus.cc:1261] T d97d6030164e453aa0f6440e85426695 P 7f8b53fbbb804e189ce714679eb3e35c [term 1 FOLLOWER]: Refusing update from remote peer fb063f3bf2c24ca29f5480118eaa718b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:43:17.985033 17293 raft_consensus.cc:1261] T d97d6030164e453aa0f6440e85426695 P 1671965ee3334946868d67415747aba5 [term 1 FOLLOWER]: Refusing update from remote peer fb063f3bf2c24ca29f5480118eaa718b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:43:17.985246 17458 consensus_queue.cc:1035] T d97d6030164e453aa0f6440e85426695 P fb063f3bf2c24ca29f5480118eaa718b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f8b53fbbb804e189ce714679eb3e35c" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 44523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:43:17.985363 17417 consensus_queue.cc:1035] T d97d6030164e453aa0f6440e85426695 P fb063f3bf2c24ca29f5480118eaa718b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1671965ee3334946868d67415747aba5" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 35545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:43:17.997080 16855 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:43:17.997184 16855 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:43:17.997886 16855 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0c41cc0a857d4203a8750d8a6b304591: Sending DeleteTablet for 3 replicas of tablet d97d6030164e453aa0f6440e85426695
I20260928 14:43:17.998165 16992 tablet_service.cc:1514] Processing DeleteTablet for tablet d97d6030164e453aa0f6440e85426695 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:43:17 UTC) from {username='oryx', principal='oryx/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:54694
I20260928 14:43:17.998206 17133 tablet_service.cc:1514] Processing DeleteTablet for tablet d97d6030164e453aa0f6440e85426695 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:43:17 UTC) from {username='oryx', principal='oryx/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:39600
I20260928 14:43:17.998207 17273 tablet_service.cc:1514] Processing DeleteTablet for tablet d97d6030164e453aa0f6440e85426695 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:43:17 UTC) from {username='oryx', principal='oryx/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:36406
I20260928 14:43:17.998263 17451 tablet_replica.cc:330] T d97d6030164e453aa0f6440e85426695 P fb063f3bf2c24ca29f5480118eaa718b: stopping tablet replica
I20260928 14:43:17.998301 17453 tablet_replica.cc:330] T d97d6030164e453aa0f6440e85426695 P 7f8b53fbbb804e189ce714679eb3e35c: stopping tablet replica
I20260928 14:43:17.998313 17451 raft_consensus.cc:2229] T d97d6030164e453aa0f6440e85426695 P fb063f3bf2c24ca29f5480118eaa718b [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:43:17.998348 17453 raft_consensus.cc:2229] T d97d6030164e453aa0f6440e85426695 P 7f8b53fbbb804e189ce714679eb3e35c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:43:17.998382 17453 raft_consensus.cc:2258] T d97d6030164e453aa0f6440e85426695 P 7f8b53fbbb804e189ce714679eb3e35c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:43:17.998369 17452 tablet_replica.cc:330] T d97d6030164e453aa0f6440e85426695 P 1671965ee3334946868d67415747aba5: stopping tablet replica
I20260928 14:43:17.998386 17451 raft_consensus.cc:2258] T d97d6030164e453aa0f6440e85426695 P fb063f3bf2c24ca29f5480118eaa718b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:43:17.998428 17452 raft_consensus.cc:2229] T d97d6030164e453aa0f6440e85426695 P 1671965ee3334946868d67415747aba5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:43:17.998467 17452 raft_consensus.cc:2258] T d97d6030164e453aa0f6440e85426695 P 1671965ee3334946868d67415747aba5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:43:17.998693 17453 ts_tablet_manager.cc:1905] T d97d6030164e453aa0f6440e85426695 P 7f8b53fbbb804e189ce714679eb3e35c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:43:17.998692 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 16928
I20260928 14:43:17.998916 17452 ts_tablet_manager.cc:1905] T d97d6030164e453aa0f6440e85426695 P 1671965ee3334946868d67415747aba5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:43:18.000727 17452 ts_tablet_manager.cc:1918] T d97d6030164e453aa0f6440e85426695 P 1671965ee3334946868d67415747aba5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:43:18.000769 17453 ts_tablet_manager.cc:1918] T d97d6030164e453aa0f6440e85426695 P 7f8b53fbbb804e189ce714679eb3e35c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260928 14:43:18.000808 17452 log.cc:1196] T d97d6030164e453aa0f6440e85426695 P 1671965ee3334946868d67415747aba5: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-2/wal/wals/d97d6030164e453aa0f6440e85426695
I20260928 14:43:18.000830 17453 log.cc:1196] T d97d6030164e453aa0f6440e85426695 P 7f8b53fbbb804e189ce714679eb3e35c: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790606561545179-9349-0/minicluster-data/ts-1/wal/wals/d97d6030164e453aa0f6440e85426695
I20260928 14:43:18.001055 17452 ts_tablet_manager.cc:1939] T d97d6030164e453aa0f6440e85426695 P 1671965ee3334946868d67415747aba5: Deleting consensus metadata
I20260928 14:43:18.001067 17453 ts_tablet_manager.cc:1939] T d97d6030164e453aa0f6440e85426695 P 7f8b53fbbb804e189ce714679eb3e35c: Deleting consensus metadata
I20260928 14:43:18.001422 16840 catalog_manager.cc:4841] TS 1671965ee3334946868d67415747aba5 (127.9.33.67:35545): tablet d97d6030164e453aa0f6440e85426695 (table test-table [id=154e79b59b0c4603a4d3145a169e710c]) successfully deleted
I20260928 14:43:18.001608 16841 catalog_manager.cc:4841] TS 7f8b53fbbb804e189ce714679eb3e35c (127.9.33.66:44523): tablet d97d6030164e453aa0f6440e85426695 (table test-table [id=154e79b59b0c4603a4d3145a169e710c]) successfully deleted
W20260928 14:43:18.007558 16842 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.33.65:33699 (error 108)
I20260928 14:43:18.007789 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 17068
I20260928 14:43:18.010092 17466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:18.009046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:33699 (local address 127.0.0.1:54734)
0928 14:43:18.009957 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:18.010027 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.65:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":884,"thread_start_us":51,"threads_started":1}
W20260928 14:43:18.010263 16842 catalog_manager.cc:4584] TS fb063f3bf2c24ca29f5480118eaa718b (127.9.33.65:33699): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d97d6030164e453aa0f6440e85426695: Network error: Client connection negotiation failed: client connection to 127.9.33.65:33699: connect: Connection refused (error 111)
I20260928 14:43:18.014832 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 17208
I20260928 14:43:18.024745 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 16824
2026-09-28T14:43:18Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3179 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:43:18 dist-test-slave-pv0w krb5kdc[17469](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:43:18 dist-test-slave-pv0w krb5kdc[17469](info): set up 2 sockets
Sep 28 14:43:18 dist-test-slave-pv0w krb5kdc[17469](info): commencing operation
krb5kdc: starting...
W20260928 14:43:19.081199 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:43:19 dist-test-slave-pv0w krb5kdc[17469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606599, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:43:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:43:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.33.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
I20260928 14:43:19.215835 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:45013
--webserver_interface=127.9.33.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:34163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:45013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:43:19.299319 17482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:19.299612 17482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:19.299656 17482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:19.300985 17482 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:43:19.301076 17482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:19.301101 17482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:19.301123 17482 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:43:19.301151 17482 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:43:19.302853 17482 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:34163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:45013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:45013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:19.303180 17482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:19.303452 17482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:19.308626 17487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:19.308629 17488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:19.308629 17490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:19.308950 17482 server_base.cc:1034] running on GCE node
I20260928 14:43:19.309245 17482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:19.309574 17482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:19.310772 17482 hybrid_clock.cc:648] HybridClock initialized: now 1790606599310763 us; error 39 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-28T14:43:19Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1283 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:43:19 dist-test-slave-pv0w krb5kdc[17495](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:43:19 dist-test-slave-pv0w krb5kdc[17495](info): set up 2 sockets
Sep 28 14:43:19 dist-test-slave-pv0w krb5kdc[17495](info): commencing operation
krb5kdc: starting...
W20260928 14:43:20.377349 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:43:20 dist-test-slave-pv0w krb5kdc[17495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:43:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:43:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.33.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
I20260928 14:43:20.503765 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:37447
--webserver_interface=127.9.33.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:42273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:37447
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:43:20.588351 17508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:20.588609 17508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:20.588649 17508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:20.589849 17508 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:43:20.589929 17508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:20.589955 17508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:20.590009 17508 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:43:20.590039 17508 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:43:20.592069 17508 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:42273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:37447
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:37447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:20.592360 17508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:20.592556 17508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:20.597680 17514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:20.597692 17513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:20.597664 17516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:20.597913 17508 server_base.cc:1034] running on GCE node
I20260928 14:43:20.598235 17508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:20.598636 17508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:20.599871 17508 hybrid_clock.cc:648] HybridClock initialized: now 1790606600599845 us; error 37 us; skew 500 ppm
Sep 28 14:43:20 dist-test-slave-pv0w krb5kdc[17495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606600, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:20.607055 17508 init.cc:377] Logged in from keytab as kudu/127.9.33.126@KRBTEST.COM (short username kudu)
I20260928 14:43:20.607596 17508 webserver.cc:460] Webserver started at http://127.9.33.126:42049/ using document root <none> and password file <none>
I20260928 14:43:20.607915 17508 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:20.607990 17508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:20.608153 17508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:20.609385 17508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "b6d7f43688494ce3b5ae1b87516002fe"
format_stamp: "Formatted at 2026-09-28 14:43:20 on dist-test-slave-pv0w"
server_key: "3000854312ad3b11f823a1f56d62f13a"
server_key_iv: "1fc517c7d7e5f55ea4c4ddc5271f022b"
server_key_version: "encryptionkey@0"
I20260928 14:43:20.609776 17508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "b6d7f43688494ce3b5ae1b87516002fe"
format_stamp: "Formatted at 2026-09-28 14:43:20 on dist-test-slave-pv0w"
server_key: "3000854312ad3b11f823a1f56d62f13a"
server_key_iv: "1fc517c7d7e5f55ea4c4ddc5271f022b"
server_key_version: "encryptionkey@0"
I20260928 14:43:20.611279 17508 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:43:20.612156 17523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:20.612360 17508 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260928 14:43:20.612646 17508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "b6d7f43688494ce3b5ae1b87516002fe"
format_stamp: "Formatted at 2026-09-28 14:43:20 on dist-test-slave-pv0w"
server_key: "3000854312ad3b11f823a1f56d62f13a"
server_key_iv: "1fc517c7d7e5f55ea4c4ddc5271f022b"
server_key_version: "encryptionkey@0"
I20260928 14:43:20.612768 17508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:20.627782 17508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:20.630300 17508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:20.630535 17508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:20.635821 17508 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:37447
I20260928 14:43:20.635900 17575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:37447 every 8 connection(s)
I20260928 14:43:20.636296 17508 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:43:20.637310 17576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:20.639249 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 17508
I20260928 14:43:20.639408 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:43:20.639611 9349 external_mini_cluster.cc:1428] Setting key 1a2aaf693887113bd2098bdf4748db10
I20260928 14:43:20.639997 17576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe: Bootstrap starting.
I20260928 14:43:20.640789 17576 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:20.641113 17576 log.cc:824] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe: Log is configured to *not* fsync() on all Append() calls
I20260928 14:43:20.642014 17576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe: No bootstrap required, opened a new log
I20260928 14:43:20.644335 17576 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d7f43688494ce3b5ae1b87516002fe" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37447 } }
I20260928 14:43:20.644517 17576 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:20.644542 17576 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6d7f43688494ce3b5ae1b87516002fe, State: Initialized, Role: FOLLOWER
I20260928 14:43:20.644698 17576 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d7f43688494ce3b5ae1b87516002fe" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37447 } }
I20260928 14:43:20.644775 17576 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:43:20.644806 17576 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:43:20.644856 17576 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:20.645689 17576 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d7f43688494ce3b5ae1b87516002fe" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37447 } }
I20260928 14:43:20.645830 17576 leader_election.cc:302] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6d7f43688494ce3b5ae1b87516002fe; no voters:
I20260928 14:43:20.646035 17576 leader_election.cc:288] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:43:20.646117 17581 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:43:20.646333 17581 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe [term 1 LEADER]: Becoming Leader. State: Replica: b6d7f43688494ce3b5ae1b87516002fe, State: Running, Role: LEADER
I20260928 14:43:20.646365 17576 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:43:20.646512 17581 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d7f43688494ce3b5ae1b87516002fe" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37447 } }
I20260928 14:43:20.646916 17582 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6d7f43688494ce3b5ae1b87516002fe. Latest consensus state: current_term: 1 leader_uuid: "b6d7f43688494ce3b5ae1b87516002fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d7f43688494ce3b5ae1b87516002fe" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37447 } } }
I20260928 14:43:20.646919 17583 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6d7f43688494ce3b5ae1b87516002fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d7f43688494ce3b5ae1b87516002fe" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37447 } } }
I20260928 14:43:20.647131 17583 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe [sys.catalog]: This master's current role is: LEADER
I20260928 14:43:20.647117 17582 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe [sys.catalog]: This master's current role is: LEADER
I20260928 14:43:20.647437 17588 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 28 14:43:20 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:43:20.648097 17588 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:43:20.651697 17579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:20.640299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43024 (local address 127.9.33.126:37447)
0928 14:43:20.640494 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:20.640502 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:20.640529 (+ 27us) server_negotiation.cc:407] Connection header received
0928 14:43:20.641070 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:20.641079 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:20.641136 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:20.641311 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:20.642172 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:20.643141 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:20.644352 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:20.645036 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:20.645129 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:20.648159 (+ 3030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:20.648165 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:20.648165 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:20.648180 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:20.649925 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:20.650376 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:20.650379 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:20.650379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:20.650492 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:20.650724 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:20.650725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:20.650725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:20.650993 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:20.651099 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:20.651269 (+ 170us) server_negotiation.cc:299] Negotiation successful
0928 14:43:20.651410 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":105,"threads_started":1}
I20260928 14:43:20.653499 17588 catalog_manager.cc:1347] Generated new cluster ID: 475d942005f146a7b9200a8611309df7
I20260928 14:43:20.653573 17588 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:43:20.668035 17588 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:43:20.668805 17588 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:43:20.674530 17588 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe: Generated new TSK 0
I20260928 14:43:20.674783 17588 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.33.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab.
Entry for principal kudu/127.9.33.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab.
I20260928 14:43:20.707155 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.65:0
--local_ip_for_outbound_sockets=127.9.33.65
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:37447
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:42273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:43:20.797046 17602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:20.797281 17602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:20.797310 17602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:20.798496 17602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:20.798539 17602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:20.798568 17602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.65
I20260928 14:43:20.800034 17602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:42273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:37447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.65
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:20.800237 17602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:20.800436 17602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:20.806030 17607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:20.806044 17608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:20.806030 17610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:20.806434 17602 server_base.cc:1034] running on GCE node
I20260928 14:43:20.806672 17602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:20.806901 17602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:20.808100 17602 hybrid_clock.cc:648] HybridClock initialized: now 1790606600808094 us; error 42 us; skew 500 ppm
Sep 28 14:43:20 dist-test-slave-pv0w krb5kdc[17495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606600, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:20.811080 17602 init.cc:377] Logged in from keytab as kudu/127.9.33.65@KRBTEST.COM (short username kudu)
I20260928 14:43:20.811632 17602 webserver.cc:460] Webserver started at http://127.9.33.65:46637/ using document root <none> and password file <none>
I20260928 14:43:20.811923 17602 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:20.812007 17602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:20.812180 17602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:20.813395 17602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-0/data/instance:
uuid: "9a8c433bfc89466e992fa2d81361c9b9"
format_stamp: "Formatted at 2026-09-28 14:43:20 on dist-test-slave-pv0w"
server_key: "6e0ddef23e45f6a66e4e0da1b4c1c53f"
server_key_iv: "636c50e2621315a5365dd69163f5a8d5"
server_key_version: "encryptionkey@0"
I20260928 14:43:20.813818 17602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance:
uuid: "9a8c433bfc89466e992fa2d81361c9b9"
format_stamp: "Formatted at 2026-09-28 14:43:20 on dist-test-slave-pv0w"
server_key: "6e0ddef23e45f6a66e4e0da1b4c1c53f"
server_key_iv: "636c50e2621315a5365dd69163f5a8d5"
server_key_version: "encryptionkey@0"
I20260928 14:43:20.815485 17602 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:43:20.816537 17617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:20.816754 17602 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:20.816869 17602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-0/wal
uuid: "9a8c433bfc89466e992fa2d81361c9b9"
format_stamp: "Formatted at 2026-09-28 14:43:20 on dist-test-slave-pv0w"
server_key: "6e0ddef23e45f6a66e4e0da1b4c1c53f"
server_key_iv: "636c50e2621315a5365dd69163f5a8d5"
server_key_version: "encryptionkey@0"
I20260928 14:43:20.816978 17602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:20.826017 17602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:20.828665 17602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:20.828851 17602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:20.829123 17602 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:43:20.829457 17602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:43:20.829514 17602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:20.829560 17602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:43:20.829586 17602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:20.836045 17602 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.65:45747
I20260928 14:43:20.836161 17730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.65:45747 every 8 connection(s)
I20260928 14:43:20.836499 17602 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
I20260928 14:43:20.842770 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 17602
I20260928 14:43:20.842881 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance
I20260928 14:43:20.843124 9349 external_mini_cluster.cc:1428] Setting key 4427f4d8146fdc8c4464278b9eebef15
Sep 28 14:43:20 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606600, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:43:20.849062 17579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:20.837369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:58703 (local address 127.9.33.126:37447)
0928 14:43:20.837439 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:20.837441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:20.837624 (+ 183us) server_negotiation.cc:407] Connection header received
0928 14:43:20.838129 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:20.838131 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:20.838142 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:20.838220 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:20.839334 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:20.840026 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:20.841227 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:20.841472 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:20.841544 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:20.844520 (+ 2976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:20.844522 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:20.844523 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:20.844536 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:20.846800 (+ 2264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:20.847415 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:20.847417 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:20.847418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:20.847608 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:20.848108 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:20.848109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:20.848110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:20.848293 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:20.848366 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:20.848883 (+ 517us) server_negotiation.cc:299] Negotiation successful
0928 14:43:20.848986 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260928 14:43:20.849303 17733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:20.837306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.65:58703)
0928 14:43:20.837478 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:20.837524 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:20.838022 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:20.838278 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:20.838282 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:20.838973 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:20.839193 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:20.839200 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:20.840155 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:20.840156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:20.841096 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:20.841099 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:20.841591 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:20.841592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:20.841739 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:20.842367 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:20.842377 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:20.844408 (+ 2031us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:20.846907 (+ 2499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:20.846920 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:20.846924 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:20.847305 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:20.847754 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:20.847755 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:20.847756 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:20.847987 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:20.848389 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:20.848399 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:20.848631 (+ 232us) client_negotiation.cc:769] Sending connection context
0928 14:43:20.848777 (+ 146us) client_negotiation.cc:240] Negotiation successful
0928 14:43:20.849006 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":91,"threads_started":1}
I20260928 14:43:20.850174 17731 heartbeater.cc:347] Connected to a master server at 127.9.33.126:37447
I20260928 14:43:20.850306 17731 heartbeater.cc:464] Registering TS with master...
I20260928 14:43:20.850569 17731 heartbeater.cc:511] Master 127.9.33.126:37447 requested a full tablet report, sending...
I20260928 14:43:20.851271 17540 ts_manager.cc:194] Registered new tserver with Master: 9a8c433bfc89466e992fa2d81361c9b9 (127.9.33.65:45747)
I20260928 14:43:20.852335 17540 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.65@KRBTEST.COM'} at 127.9.33.65:58703
WARNING: no policy specified for kudu/127.9.33.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab.
Entry for principal kudu/127.9.33.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab.
I20260928 14:43:20.877204 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.66:0
--local_ip_for_outbound_sockets=127.9.33.66
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:37447
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:42273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:43:20.961462 17736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:20.961694 17736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:20.961720 17736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:20.962883 17736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:20.962927 17736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:20.962955 17736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.66
I20260928 14:43:20.964429 17736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:42273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:37447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.66
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:20.964648 17736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:20.964844 17736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:20.970474 17741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:20.970465 17744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:20.970873 17742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:20.970901 17736 server_base.cc:1034] running on GCE node
I20260928 14:43:20.971156 17736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:20.971421 17736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:20.972599 17736 hybrid_clock.cc:648] HybridClock initialized: now 1790606600972588 us; error 41 us; skew 500 ppm
Sep 28 14:43:20 dist-test-slave-pv0w krb5kdc[17495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606600, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:20.975219 17736 init.cc:377] Logged in from keytab as kudu/127.9.33.66@KRBTEST.COM (short username kudu)
I20260928 14:43:20.975633 17736 webserver.cc:460] Webserver started at http://127.9.33.66:37597/ using document root <none> and password file <none>
I20260928 14:43:20.975889 17736 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:20.975939 17736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:20.976032 17736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:20.977094 17736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-1/data/instance:
uuid: "71030387d0044529b20214576940670a"
format_stamp: "Formatted at 2026-09-28 14:43:20 on dist-test-slave-pv0w"
server_key: "72be6bc80ef3ef0afe15b7cf7764f392"
server_key_iv: "3348fdef9ed9c669b4ed628c0cf34c92"
server_key_version: "encryptionkey@0"
I20260928 14:43:20.977406 17736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance:
uuid: "71030387d0044529b20214576940670a"
format_stamp: "Formatted at 2026-09-28 14:43:20 on dist-test-slave-pv0w"
server_key: "72be6bc80ef3ef0afe15b7cf7764f392"
server_key_iv: "3348fdef9ed9c669b4ed628c0cf34c92"
server_key_version: "encryptionkey@0"
I20260928 14:43:20.978761 17736 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:43:20.979604 17751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:20.979879 17736 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:20.979971 17736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-1/wal
uuid: "71030387d0044529b20214576940670a"
format_stamp: "Formatted at 2026-09-28 14:43:20 on dist-test-slave-pv0w"
server_key: "72be6bc80ef3ef0afe15b7cf7764f392"
server_key_iv: "3348fdef9ed9c669b4ed628c0cf34c92"
server_key_version: "encryptionkey@0"
I20260928 14:43:20.980034 17736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:20.996747 17736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:20.998818 17736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:20.998940 17736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:20.999150 17736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:43:20.999425 17736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:43:20.999455 17736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:20.999478 17736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:43:20.999490 17736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:21.005388 17736 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.66:37943
I20260928 14:43:21.005467 17864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.66:37943 every 8 connection(s)
I20260928 14:43:21.005806 17736 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
Sep 28 14:43:21 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606600, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:43:21.014202 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 17736
I20260928 14:43:21.014343 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance
I20260928 14:43:21.014564 9349 external_mini_cluster.cc:1428] Setting key 589441e224d9c520d43f9de55d4ed9b8
I20260928 14:43:21.016739 17579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:21.006646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:38819 (local address 127.9.33.126:37447)
0928 14:43:21.006710 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:21.006712 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:21.006891 (+ 179us) server_negotiation.cc:407] Connection header received
0928 14:43:21.007383 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:21.007384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:21.007394 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:21.007459 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:21.008622 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.009166 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.010198 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.010397 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.010464 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:21.012737 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:21.012739 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:21.012739 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:21.012747 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:21.015009 (+ 2262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:21.015518 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:21.015520 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:21.015520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:21.015576 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:21.015945 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:21.015946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:21.015946 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:21.016102 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:21.016174 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:21.016574 (+ 400us) server_negotiation.cc:299] Negotiation successful
0928 14:43:21.016667 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260928 14:43:21.017010 17867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:21.006565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.66:38819)
0928 14:43:21.006756 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:21.006800 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:21.007299 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:21.007525 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:21.007529 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:21.008228 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:21.008499 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:21.008505 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.009261 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.009261 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:21.010094 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:21.010096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.010480 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.010481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:21.010584 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:21.011088 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:21.011097 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:21.012625 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:21.015076 (+ 2451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:21.015084 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:21.015087 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:21.015433 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:21.015647 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:21.015647 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:21.015648 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:21.015834 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:21.016172 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:21.016179 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:21.016382 (+ 203us) client_negotiation.cc:769] Sending connection context
0928 14:43:21.016497 (+ 115us) client_negotiation.cc:240] Negotiation successful
0928 14:43:21.016693 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":93,"threads_started":1}
I20260928 14:43:21.017673 17865 heartbeater.cc:347] Connected to a master server at 127.9.33.126:37447
I20260928 14:43:21.017786 17865 heartbeater.cc:464] Registering TS with master...
I20260928 14:43:21.018015 17865 heartbeater.cc:511] Master 127.9.33.126:37447 requested a full tablet report, sending...
I20260928 14:43:21.018468 17540 ts_manager.cc:194] Registered new tserver with Master: 71030387d0044529b20214576940670a (127.9.33.66:37943)
I20260928 14:43:21.018890 17540 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.66@KRBTEST.COM'} at 127.9.33.66:38819
WARNING: no policy specified for kudu/127.9.33.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab.
Entry for principal kudu/127.9.33.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab.
I20260928 14:43:21.047179 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.67:0
--local_ip_for_outbound_sockets=127.9.33.67
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:37447
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:42273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:43:21.130564 17870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:21.130841 17870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:21.130883 17870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:21.132344 17870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:21.132422 17870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:21.132472 17870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.67
I20260928 14:43:21.134094 17870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:42273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:37447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.67
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:21.134402 17870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:21.134693 17870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:21.140056 17878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:21.140044 17875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:21.140305 17870 server_base.cc:1034] running on GCE node
W20260928 14:43:21.140244 17876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:21.140594 17870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:21.140853 17870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:21.142050 17870 hybrid_clock.cc:648] HybridClock initialized: now 1790606601142026 us; error 41 us; skew 500 ppm
Sep 28 14:43:21 dist-test-slave-pv0w krb5kdc[17495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606601, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:21.144914 17870 init.cc:377] Logged in from keytab as kudu/127.9.33.67@KRBTEST.COM (short username kudu)
I20260928 14:43:21.145494 17870 webserver.cc:460] Webserver started at http://127.9.33.67:33327/ using document root <none> and password file <none>
I20260928 14:43:21.145743 17870 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:21.145817 17870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:21.145982 17870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:21.147308 17870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-2/data/instance:
uuid: "cdea984308f54835a6f10942b86dca4d"
format_stamp: "Formatted at 2026-09-28 14:43:21 on dist-test-slave-pv0w"
server_key: "853913bb55bddd1134d6f4655e2b6a3a"
server_key_iv: "22e1a33f19630b156c1f383dc74d6da3"
server_key_version: "encryptionkey@0"
I20260928 14:43:21.147902 17870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance:
uuid: "cdea984308f54835a6f10942b86dca4d"
format_stamp: "Formatted at 2026-09-28 14:43:21 on dist-test-slave-pv0w"
server_key: "853913bb55bddd1134d6f4655e2b6a3a"
server_key_iv: "22e1a33f19630b156c1f383dc74d6da3"
server_key_version: "encryptionkey@0"
I20260928 14:43:21.149688 17870 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:21.150581 17885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:21.150826 17870 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:21.150940 17870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-2/wal
uuid: "cdea984308f54835a6f10942b86dca4d"
format_stamp: "Formatted at 2026-09-28 14:43:21 on dist-test-slave-pv0w"
server_key: "853913bb55bddd1134d6f4655e2b6a3a"
server_key_iv: "22e1a33f19630b156c1f383dc74d6da3"
server_key_version: "encryptionkey@0"
I20260928 14:43:21.151050 17870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:21.186877 17870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:21.189284 17870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:21.189567 17870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:21.189834 17870 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:43:21.190177 17870 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:43:21.190234 17870 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:21.190281 17870 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:43:21.190305 17870 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:21.196564 17870 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.67:34519
I20260928 14:43:21.196612 17998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.67:34519 every 8 connection(s)
I20260928 14:43:21.196986 17870 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
Sep 28 14:43:21 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606601, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:43:21.203445 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 17870
I20260928 14:43:21.203536 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance
I20260928 14:43:21.203804 9349 external_mini_cluster.cc:1428] Setting key af1339917f97f73b1efcde4f74014010
I20260928 14:43:21.206777 17579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:21.197775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:38011 (local address 127.9.33.126:37447)
0928 14:43:21.197846 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:21.197848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:21.198066 (+ 218us) server_negotiation.cc:407] Connection header received
0928 14:43:21.198550 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:21.198551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:21.198563 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:21.198609 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:21.199566 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.200143 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.201182 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.201380 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.201438 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:21.203775 (+ 2337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:21.203776 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:21.203777 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:21.203786 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:21.205312 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:21.205744 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:21.205746 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:21.205747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:21.205807 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:21.206089 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:21.206090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:21.206090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:21.206212 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:21.206290 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:21.206593 (+ 303us) server_negotiation.cc:299] Negotiation successful
0928 14:43:21.206678 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260928 14:43:21.206995 18001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:21.197775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.67:38011)
0928 14:43:21.197937 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:21.197975 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:21.198466 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:21.198674 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:21.198677 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:21.199253 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:21.199470 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:21.199475 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.200221 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.200222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:21.201089 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:21.201091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.201493 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.201494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:21.201617 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:21.202106 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:21.202116 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:21.203643 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:21.205416 (+ 1773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:21.205423 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:21.205426 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:21.205665 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:21.205885 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:21.205886 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:21.205886 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:21.206013 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:21.206283 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:21.206289 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:21.206455 (+ 166us) client_negotiation.cc:769] Sending connection context
0928 14:43:21.206570 (+ 115us) client_negotiation.cc:240] Negotiation successful
0928 14:43:21.206745 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":94,"threads_started":1}
I20260928 14:43:21.207615 17999 heartbeater.cc:347] Connected to a master server at 127.9.33.126:37447
I20260928 14:43:21.207830 17999 heartbeater.cc:464] Registering TS with master...
I20260928 14:43:21.208067 17999 heartbeater.cc:511] Master 127.9.33.126:37447 requested a full tablet report, sending...
I20260928 14:43:21.208544 17540 ts_manager.cc:194] Registered new tserver with Master: cdea984308f54835a6f10942b86dca4d (127.9.33.67:34519)
I20260928 14:43:21.208943 17540 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.67@KRBTEST.COM'} at 127.9.33.67:38011
I20260928 14:43:21.215859 9349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.9.33.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.33.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.33.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.65.keytab.
Entry for principal oryx/127.9.33.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.65.keytab.
WARNING: no policy specified for oryx/127.9.33.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.33.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.33.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.66.keytab.
Entry for principal oryx/127.9.33.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.66.keytab.
WARNING: no policy specified for oryx/127.9.33.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.33.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.33.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.67.keytab.
Entry for principal oryx/127.9.33.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.67.keytab.
I20260928 14:43:21.295323 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 17602
I20260928 14:43:21.300057 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 17736
I20260928 14:43:21.304749 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 17870
I20260928 14:43:21.309610 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 17508
I20260928 14:43:21.315063 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:37447
--webserver_interface=127.9.33.126
--webserver_port=42049
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:42273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:37447
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:43:21.401613 18008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:21.401855 18008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:21.401882 18008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:21.403036 18008 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:43:21.403077 18008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:21.403092 18008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:21.403105 18008 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:43:21.403117 18008 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:43:21.404692 18008 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:42273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:37447
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:37447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=42049
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:21.404927 18008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:21.405114 18008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:21.410385 18014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:21.410565 18008 server_base.cc:1034] running on GCE node
W20260928 14:43:21.410385 18016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:21.410392 18013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:21.411171 18008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:21.411502 18008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:21.412720 18008 hybrid_clock.cc:648] HybridClock initialized: now 1790606601412717 us; error 47 us; skew 500 ppm
Sep 28 14:43:21 dist-test-slave-pv0w krb5kdc[17495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606601, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:21.415504 18008 init.cc:377] Logged in from keytab as kudu/127.9.33.126@KRBTEST.COM (short username kudu)
I20260928 14:43:21.416047 18008 webserver.cc:460] Webserver started at http://127.9.33.126:42049/ using document root <none> and password file <none>
I20260928 14:43:21.416293 18008 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:21.416388 18008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:21.417934 18008 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:43:21.418717 18023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:21.418946 18008 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:21.419056 18008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "b6d7f43688494ce3b5ae1b87516002fe"
format_stamp: "Formatted at 2026-09-28 14:43:20 on dist-test-slave-pv0w"
server_key: "3000854312ad3b11f823a1f56d62f13a"
server_key_iv: "1fc517c7d7e5f55ea4c4ddc5271f022b"
server_key_version: "encryptionkey@0"
I20260928 14:43:21.419450 18008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:21.429863 18008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:21.432265 18008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:21.432453 18008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:21.436697 18008 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:37447
I20260928 14:43:21.436733 18075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:37447 every 8 connection(s)
I20260928 14:43:21.437150 18008 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:43:21.438751 18076 sys_catalog.cc:263] Verifying existing consensus state
I20260928 14:43:21.439486 18076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe: Bootstrap starting.
I20260928 14:43:21.440514 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 18008
I20260928 14:43:21.440816 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.65:45747
--local_ip_for_outbound_sockets=127.9.33.65
--tserver_master_addrs=127.9.33.126:37447
--webserver_port=46637
--webserver_interface=127.9.33.65
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:42273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.33.65
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.65.keytab with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:43:21.448565 18076 log.cc:824] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe: Log is configured to *not* fsync() on all Append() calls
I20260928 14:43:21.450387 18076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 14:43:21.450706 18076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe: Bootstrap complete.
I20260928 14:43:21.453099 18076 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d7f43688494ce3b5ae1b87516002fe" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37447 } }
I20260928 14:43:21.453331 18076 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6d7f43688494ce3b5ae1b87516002fe, State: Initialized, Role: FOLLOWER
I20260928 14:43:21.453519 18076 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d7f43688494ce3b5ae1b87516002fe" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37447 } }
I20260928 14:43:21.453585 18076 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:43:21.453652 18076 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:43:21.453701 18076 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe [term 1 FOLLOWER]: Advancing to term 2
I20260928 14:43:21.454810 18076 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d7f43688494ce3b5ae1b87516002fe" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37447 } }
I20260928 14:43:21.454936 18076 leader_election.cc:302] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6d7f43688494ce3b5ae1b87516002fe; no voters:
I20260928 14:43:21.455138 18076 leader_election.cc:288] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe [CANDIDATE]: Term 2 election: Requested vote from peers
I20260928 14:43:21.455188 18080 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe [term 2 FOLLOWER]: Leader election won for term 2
I20260928 14:43:21.455366 18080 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe [term 2 LEADER]: Becoming Leader. State: Replica: b6d7f43688494ce3b5ae1b87516002fe, State: Running, Role: LEADER
I20260928 14:43:21.455449 18076 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:43:21.455519 18080 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d7f43688494ce3b5ae1b87516002fe" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37447 } }
I20260928 14:43:21.455828 18080 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6d7f43688494ce3b5ae1b87516002fe. Latest consensus state: current_term: 2 leader_uuid: "b6d7f43688494ce3b5ae1b87516002fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d7f43688494ce3b5ae1b87516002fe" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37447 } } }
I20260928 14:43:21.455924 18080 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe [sys.catalog]: This master's current role is: LEADER
I20260928 14:43:21.455837 18081 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b6d7f43688494ce3b5ae1b87516002fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d7f43688494ce3b5ae1b87516002fe" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37447 } } }
I20260928 14:43:21.456100 18081 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe [sys.catalog]: This master's current role is: LEADER
I20260928 14:43:21.456563 18086 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:43:21.457129 18086 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:43:21.457314 18086 catalog_manager.cc:1259] Loaded cluster ID: 475d942005f146a7b9200a8611309df7
I20260928 14:43:21.457453 18086 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:43:21.458604 18086 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:43:21.458770 18086 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b6d7f43688494ce3b5ae1b87516002fe: Loaded TSK: 0
I20260928 14:43:21.458976 18086 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 14:43:21.525332 18078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:21.525564 18078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:21.525590 18078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:21.526829 18078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:21.526878 18078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:21.526906 18078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.65
I20260928 14:43:21.528509 18078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:42273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.65.keytab
--principal=oryx/127.9.33.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.65:45747
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.33.65
--webserver_port=46637
--tserver_master_addrs=127.9.33.126:37447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.65
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:21.528781 18078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:21.529037 18078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:21.534279 18102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:21.534304 18103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:21.534286 18105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:21.534520 18078 server_base.cc:1034] running on GCE node
I20260928 14:43:21.534801 18078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:21.535102 18078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:21.536290 18078 hybrid_clock.cc:648] HybridClock initialized: now 1790606601536272 us; error 30 us; skew 500 ppm
Sep 28 14:43:21 dist-test-slave-pv0w krb5kdc[17495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606601, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:21.539366 18078 init.cc:377] Logged in from keytab as oryx/127.9.33.65@KRBTEST.COM (short username oryx)
I20260928 14:43:21.539896 18078 webserver.cc:460] Webserver started at http://127.9.33.65:46637/ using document root <none> and password file <none>
I20260928 14:43:21.540155 18078 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:21.540242 18078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:21.541786 18078 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260928 14:43:21.542480 18112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:21.542687 18078 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:43:21.542788 18078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-0/wal
uuid: "9a8c433bfc89466e992fa2d81361c9b9"
format_stamp: "Formatted at 2026-09-28 14:43:20 on dist-test-slave-pv0w"
server_key: "6e0ddef23e45f6a66e4e0da1b4c1c53f"
server_key_iv: "636c50e2621315a5365dd69163f5a8d5"
server_key_version: "encryptionkey@0"
I20260928 14:43:21.543196 18078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:21.555032 18078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:21.557781 18078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:21.557961 18078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:21.558208 18078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:43:21.558522 18078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:43:21.558576 18078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:21.558619 18078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:43:21.558646 18078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:21.565742 18078 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.65:45747
I20260928 14:43:21.565789 18225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.65:45747 every 8 connection(s)
I20260928 14:43:21.566164 18078 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
I20260928 14:43:21.566465 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 18078
I20260928 14:43:21.566771 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.66:37943
--local_ip_for_outbound_sockets=127.9.33.66
--tserver_master_addrs=127.9.33.126:37447
--webserver_port=37597
--webserver_interface=127.9.33.66
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:42273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.33.66
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.66.keytab with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 14:43:21 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:21 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:21.583261 18229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:21.572560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.65:57979)
0928 14:43:21.572793 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:21.572844 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:21.573472 (+ 628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:21.574105 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:21.574109 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:21.574913 (+ 804us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:21.575193 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:21.575200 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.577156 (+ 1956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.577158 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:21.578393 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:21.578396 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.579261 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.579262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:21.579439 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:21.580130 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:21.580161 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:21.583054 (+ 2893us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":97,"threads_started":1}
W20260928 14:43:21.583498 18226 heartbeater.cc:644] Failed to heartbeat to 127.9.33.126:37447 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.33.126:37447: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
W20260928 14:43:21.583707 18230 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:21.572568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:57979 (local address 127.9.33.126:37447)
0928 14:43:21.572905 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:21.572916 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:21.573054 (+ 138us) server_negotiation.cc:407] Connection header received
0928 14:43:21.573737 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:21.573746 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:21.573875 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:21.574054 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:21.575338 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.577025 (+ 1687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.578495 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.579146 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.579258 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:21.583228 (+ 3970us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.65:57979: BlockingRecv error: recv got EOF from 127.9.33.65:57979 (error 108)
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":102,"threads_started":1}
Sep 28 14:43:21 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:21 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:21.589425 18229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:21.583690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.65:42137)
0928 14:43:21.584062 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:21.584075 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:21.584130 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:21.584287 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:21.584288 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:21.584439 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:21.584502 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:21.584503 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.585436 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.585437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:21.586267 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:21.586269 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.586736 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.586737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:21.586800 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:21.587283 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:21.587290 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:21.589343 (+ 2053us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":261,"threads_started":1}
W20260928 14:43:21.589509 18230 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:21.583817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:42137 (local address 127.9.33.126:37447)
0928 14:43:21.583975 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:21.583977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:21.584133 (+ 156us) server_negotiation.cc:407] Connection header received
0928 14:43:21.584199 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:21.584200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:21.584214 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:21.584267 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:21.584570 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.585343 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.586349 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.586658 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.586727 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:21.589449 (+ 2722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.65:42137: BlockingRecv error: recv got EOF from 127.9.33.65:42137 (error 108)
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":86,"threads_started":1}
Sep 28 14:43:21 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:21 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:21.595000 18232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:21.589691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.65:60199)
0928 14:43:21.589839 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:21.589849 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:21.589911 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:21.590069 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:21.590070 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:21.590236 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:21.590292 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:21.590293 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.591241 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.591242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:21.592008 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:21.592010 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.592447 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.592448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:21.592493 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:21.592959 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:21.592963 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:21.594907 (+ 1944us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125}
W20260928 14:43:21.595183 18226 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 14:43:21.595221 18230 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:21.589770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:60199 (local address 127.9.33.126:37447)
0928 14:43:21.589805 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:21.589807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:21.589915 (+ 108us) server_negotiation.cc:407] Connection header received
0928 14:43:21.589968 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:21.589969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:21.589981 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:21.590053 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:21.590419 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.591147 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.592080 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.592359 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.592431 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:21.595141 (+ 2710us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.65:60199: BlockingRecv error: recv got EOF from 127.9.33.65:60199 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260928 14:43:21.656672 18228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:21.656898 18228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:21.656924 18228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:21.658243 18228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:21.658303 18228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:21.658334 18228 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.66
I20260928 14:43:21.659857 18228 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:42273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.66.keytab
--principal=oryx/127.9.33.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.66:37943
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.33.66
--webserver_port=37597
--tserver_master_addrs=127.9.33.126:37447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.66
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:21.660120 18228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:21.660377 18228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:21.665769 18237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:21.665756 18240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:21.665989 18228 server_base.cc:1034] running on GCE node
W20260928 14:43:21.665756 18238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:21.666359 18228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:21.666635 18228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:21.667801 18228 hybrid_clock.cc:648] HybridClock initialized: now 1790606601667797 us; error 83 us; skew 500 ppm
Sep 28 14:43:21 dist-test-slave-pv0w krb5kdc[17495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606601, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:21.670447 18228 init.cc:377] Logged in from keytab as oryx/127.9.33.66@KRBTEST.COM (short username oryx)
I20260928 14:43:21.670940 18228 webserver.cc:460] Webserver started at http://127.9.33.66:37597/ using document root <none> and password file <none>
I20260928 14:43:21.671197 18228 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:21.671277 18228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:21.673197 18228 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:21.673942 18247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:21.674161 18228 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:21.674256 18228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-1/wal
uuid: "71030387d0044529b20214576940670a"
format_stamp: "Formatted at 2026-09-28 14:43:20 on dist-test-slave-pv0w"
server_key: "72be6bc80ef3ef0afe15b7cf7764f392"
server_key_iv: "3348fdef9ed9c669b4ed628c0cf34c92"
server_key_version: "encryptionkey@0"
I20260928 14:43:21.674633 18228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:21.687916 18228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:21.690378 18228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:21.690526 18228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:21.690801 18228 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:43:21.691130 18228 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:43:21.691184 18228 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:21.691226 18228 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:43:21.691254 18228 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:21.697526 18228 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.66:37943
I20260928 14:43:21.697571 18360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.66:37943 every 8 connection(s)
I20260928 14:43:21.697986 18228 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
I20260928 14:43:21.702309 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 18228
I20260928 14:43:21.702615 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.67:34519
--local_ip_for_outbound_sockets=127.9.33.67
--tserver_master_addrs=127.9.33.126:37447
--webserver_port=33327
--webserver_interface=127.9.33.67
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:42273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.33.67
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.67.keytab with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 14:43:21 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:21 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
W20260928 14:43:21.706138 18230 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:21.698847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:38475 (local address 127.9.33.126:37447)
0928 14:43:21.698917 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:21.698919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:21.699133 (+ 214us) server_negotiation.cc:407] Connection header received
0928 14:43:21.699635 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:21.699637 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:21.699653 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:21.699708 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:21.700699 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.701203 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.702353 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.702605 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.702661 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:21.706080 (+ 3419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.66:38475: BlockingRecv error: recv got EOF from 127.9.33.66:38475 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260928 14:43:21.706208 18363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:21.698842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.66:38475)
0928 14:43:21.699003 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:21.699042 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:21.699540 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:21.699797 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:21.699801 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:21.700394 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:21.700610 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:21.700615 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.701312 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.701313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:21.702251 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:21.702255 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.702701 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.702702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:21.702871 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:21.703503 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:21.703514 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:21.705988 (+ 2474us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
W20260928 14:43:21.706451 18361 heartbeater.cc:644] Failed to heartbeat to 127.9.33.126:37447 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.33.126:37447: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Sep 28 14:43:21 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:21 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:21.711113 18363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:21.706705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.66:44879)
0928 14:43:21.706759 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:21.706767 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:21.706824 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:21.706991 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:21.706993 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:21.707132 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:21.707180 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:21.707181 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.707906 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.707907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:21.708418 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:21.708419 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.708793 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.708794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:21.708865 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:21.709324 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:21.709332 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:21.711025 (+ 1693us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260928 14:43:21.711169 18230 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:21.706781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:44879 (local address 127.9.33.126:37447)
0928 14:43:21.706849 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:21.706851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:21.706872 (+ 21us) server_negotiation.cc:407] Connection header received
0928 14:43:21.706910 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:21.706910 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:21.706919 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:21.706975 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:21.707258 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.707822 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.708500 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.708713 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.708792 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:21.711104 (+ 2312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.66:44879: BlockingRecv error: recv got EOF from 127.9.33.66:44879 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 28 14:43:21 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:21 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:21.716403 18363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:21.711462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.66:54341)
0928 14:43:21.711505 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:21.711516 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:21.711552 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:21.711759 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:21.711760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:21.711888 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:21.711919 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:21.711921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.712971 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.712972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:21.713475 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:21.713476 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.713959 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.713959 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:21.713998 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:21.714484 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:21.714489 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:21.716315 (+ 1826us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260928 14:43:21.716578 18230 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:21.711511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:54341 (local address 127.9.33.126:37447)
0928 14:43:21.711579 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:21.711582 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:21.711595 (+ 13us) server_negotiation.cc:407] Connection header received
0928 14:43:21.711621 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:21.711622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:21.711635 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:21.711737 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:21.712000 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.712864 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.713556 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.713860 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.713928 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:21.716431 (+ 2503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.66:54341: BlockingRecv error: recv got EOF from 127.9.33.66:54341 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260928 14:43:21.716590 18361 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 14:43:21.796832 18364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:21.797122 18364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:21.797168 18364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:21.798573 18364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:21.798653 18364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:21.798707 18364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.67
I20260928 14:43:21.800508 18364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:42273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/krb5kdc/oryx_127.9.33.67.keytab
--principal=oryx/127.9.33.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.67:34519
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.33.67
--webserver_port=33327
--tserver_master_addrs=127.9.33.126:37447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.67
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:21.800824 18364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:21.801093 18364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:21.807000 18370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:21.807032 18364 server_base.cc:1034] running on GCE node
W20260928 14:43:21.807114 18372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:21.807000 18369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:21.807511 18364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:21.807919 18364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:21.809121 18364 hybrid_clock.cc:648] HybridClock initialized: now 1790606601809094 us; error 59 us; skew 500 ppm
Sep 28 14:43:21 dist-test-slave-pv0w krb5kdc[17495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606601, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.33.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:21.811964 18364 init.cc:377] Logged in from keytab as oryx/127.9.33.67@KRBTEST.COM (short username oryx)
I20260928 14:43:21.812459 18364 webserver.cc:460] Webserver started at http://127.9.33.67:33327/ using document root <none> and password file <none>
I20260928 14:43:21.812704 18364 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:21.812798 18364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:21.814224 18364 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:21.814821 18379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:21.815196 18364 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:21.815302 18364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-2/wal
uuid: "cdea984308f54835a6f10942b86dca4d"
format_stamp: "Formatted at 2026-09-28 14:43:21 on dist-test-slave-pv0w"
server_key: "853913bb55bddd1134d6f4655e2b6a3a"
server_key_iv: "22e1a33f19630b156c1f383dc74d6da3"
server_key_version: "encryptionkey@0"
I20260928 14:43:21.815681 18364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:21.829341 18364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:21.831359 18364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:21.831547 18364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:21.831845 18364 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:43:21.832185 18364 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:43:21.832242 18364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:21.832283 18364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:43:21.832333 18364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:21.838789 18364 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.67:34519
I20260928 14:43:21.838877 18492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.67:34519 every 8 connection(s)
I20260928 14:43:21.839265 18364 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
Sep 28 14:43:21 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:21 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:21.847414 18495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:21.840088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.67:57691)
0928 14:43:21.840271 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:21.840309 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:21.840844 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:21.841105 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:21.841109 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:21.841699 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:21.841905 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:21.841910 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.842967 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.842968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:21.843928 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:21.843931 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.844292 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.844292 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:21.844401 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:21.844922 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:21.844932 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:21.847258 (+ 2326us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":86,"threads_started":1}
W20260928 14:43:21.847433 18230 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:21.840085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:57691 (local address 127.9.33.126:37447)
0928 14:43:21.840166 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:21.840168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:21.840401 (+ 233us) server_negotiation.cc:407] Connection header received
0928 14:43:21.840960 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:21.840962 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:21.840974 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:21.841021 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:21.842013 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.842835 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.844011 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.844209 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.844274 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:21.847381 (+ 3107us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.67:57691: BlockingRecv error: recv got EOF from 127.9.33.67:57691 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260928 14:43:21.847671 18493 heartbeater.cc:644] Failed to heartbeat to 127.9.33.126:37447 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.33.126:37447: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
I20260928 14:43:21.848546 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 18364
Sep 28 14:43:21 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:21 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:21.853724 18495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:21.847932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.67:43449)
0928 14:43:21.847984 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:21.847995 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:21.848039 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:21.848278 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:21.848279 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:21.848444 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:21.848507 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:21.848509 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.849477 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.849478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:21.850266 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:21.850268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.850763 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.850764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:21.850827 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:21.851305 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:21.851334 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:21.853622 (+ 2288us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260928 14:43:21.853765 18230 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:21.848134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:43449 (local address 127.9.33.126:37447)
0928 14:43:21.848166 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:21.848168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:21.848175 (+ 7us) server_negotiation.cc:407] Connection header received
0928 14:43:21.848198 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:21.848199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:21.848206 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:21.848260 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:21.848662 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.849410 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.850362 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.850665 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.850745 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:21.853717 (+ 2972us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.67:43449: BlockingRecv error: recv got EOF from 127.9.33.67:43449 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
I20260928 14:43:21.858022 18231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:21.849010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43032 (local address 127.9.33.126:37447)
0928 14:43:21.849056 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:21.849058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:21.849220 (+ 162us) server_negotiation.cc:407] Connection header received
0928 14:43:21.849298 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:21.849299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:21.849310 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:21.849375 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:21.850116 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.850830 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.852035 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.852417 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.852499 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:21.854110 (+ 1611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:21.854117 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:21.854117 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:21.854133 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:21.856310 (+ 2177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:21.856803 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:21.856807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:21.856807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:21.856952 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:21.857183 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:21.857184 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:21.857185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:21.857501 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:21.857652 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:21.857781 (+ 129us) server_negotiation.cc:299] Negotiation successful
0928 14:43:21.857907 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Sep 28 14:43:21 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:21 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:21.859992 18495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:21.854175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.67:47081)
0928 14:43:21.854287 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:21.854300 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:21.854366 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:21.854729 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:21.854730 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:21.854885 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:21.854929 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:21.854931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.855915 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.855916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:21.856802 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:21.856805 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.857206 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.857207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:21.857260 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:21.857747 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:21.857753 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:21.859938 (+ 2185us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":92}
W20260928 14:43:21.860132 18230 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:21.854235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:47081 (local address 127.9.33.126:37447)
0928 14:43:21.854303 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:21.854305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:21.854342 (+ 37us) server_negotiation.cc:407] Connection header received
0928 14:43:21.854642 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:21.854643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:21.854653 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:21.854703 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:21.855035 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.855840 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.856881 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:21.857135 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:21.857197 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:21.860048 (+ 2851us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.67:47081: BlockingRecv error: recv got EOF from 127.9.33.67:47081 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260928 14:43:21.860126 18493 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 28 14:43:22 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:22 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:22.600872 18497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:22.595639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.65:39237)
0928 14:43:22.595895 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:22.595918 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:22.595993 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:22.596184 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:22.596185 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:22.596432 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:22.596499 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:22.596501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:22.597335 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:22.597336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:22.597884 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:22.597885 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:22.598265 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:22.598266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:22.598325 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:22.598856 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:22.598861 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:22.600812 (+ 1951us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":131,"threads_started":1}
W20260928 14:43:22.601052 18498 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:22.595787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:39237 (local address 127.9.33.126:37447)
0928 14:43:22.595963 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:22.595965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:22.595975 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:43:22.596072 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:22.596074 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:22.596089 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:22.596155 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:22.596618 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:22.597250 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:22.597998 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:22.598194 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:22.598259 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:22.600963 (+ 2704us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.65:39237: BlockingRecv error: recv got EOF from 127.9.33.65:39237 (error 108)
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":84,"threads_started":1}
Sep 28 14:43:22 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:22 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:22.723053 18499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:22.717159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.66:46725)
0928 14:43:22.717372 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:22.717385 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:22.717467 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:22.717634 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:22.717635 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:22.717890 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:22.717940 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:22.717942 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:22.718967 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:22.718968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:22.719636 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:22.719639 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:22.720123 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:22.720124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:22.720178 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:22.720625 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:22.720630 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:22.722977 (+ 2347us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":96,"threads_started":1}
W20260928 14:43:22.723222 18498 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:22.717275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:46725 (local address 127.9.33.126:37447)
0928 14:43:22.717363 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:22.717366 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:22.717444 (+ 78us) server_negotiation.cc:407] Connection header received
0928 14:43:22.717536 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:22.717537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:22.717558 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:22.717629 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:22.718075 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:22.718865 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:22.719763 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:22.720043 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:22.720122 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:22.723114 (+ 2992us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.66:46725: BlockingRecv error: recv got EOF from 127.9.33.66:46725 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 28 14:43:22 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:22 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:22.865655 18500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:22.860611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.67:57557)
0928 14:43:22.860761 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:22.860778 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:22.860841 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:22.860998 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:22.860999 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:22.861167 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:22.861218 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:22.861219 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:22.862045 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:22.862046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:22.862795 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:22.862797 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:22.863237 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:22.863237 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:22.863280 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:22.863816 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:22.863820 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:22.865585 (+ 1765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
W20260928 14:43:22.865761 18498 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:22.860715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:57557 (local address 127.9.33.126:37447)
0928 14:43:22.860785 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:22.860787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:22.860851 (+ 64us) server_negotiation.cc:407] Connection header received
0928 14:43:22.860885 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:22.860886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:22.860900 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:22.860973 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:22.861305 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:22.861958 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:22.862881 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:22.863157 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:22.863233 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:22.865688 (+ 2455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.67:57557: BlockingRecv error: recv got EOF from 127.9.33.67:57557 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Sep 28 14:43:23 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:23 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:23.606730 18501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:23.601438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.65:49437)
0928 14:43:23.601591 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:23.601608 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:23.601663 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:23.601912 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:23.601913 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:23.602082 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:23.602138 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:23.602139 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:23.602858 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:23.602860 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:23.603456 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:23.603457 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:23.603852 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:23.603852 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:23.603905 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:23.604559 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:23.604565 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:23.606669 (+ 2104us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":93,"threads_started":1}
W20260928 14:43:23.606928 18502 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:23.601591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:49437 (local address 127.9.33.126:37447)
0928 14:43:23.601719 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:23.601722 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:23.601738 (+ 16us) server_negotiation.cc:407] Connection header received
0928 14:43:23.601766 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:23.601767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:23.601777 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:23.601848 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:23.602225 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:23.602752 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:23.603546 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:23.603783 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:23.603839 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:23.606795 (+ 2956us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.65:49437: BlockingRecv error: recv got EOF from 127.9.33.65:49437 (error 108)
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":73,"threads_started":1}
Sep 28 14:43:23 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:23 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:23.728437 18503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:23.723588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.66:46857)
0928 14:43:23.723765 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:23.723785 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:23.723853 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:23.724020 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:23.724020 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:23.724191 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:23.724245 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:23.724246 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:23.724926 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:23.724927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:23.725473 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:23.725475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:23.725855 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:23.725856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:23.725918 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:23.726443 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:23.726448 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:23.728375 (+ 1927us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":92,"threads_started":1}
W20260928 14:43:23.728595 18502 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:23.723691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:46857 (local address 127.9.33.126:37447)
0928 14:43:23.723763 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:23.723767 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:23.723889 (+ 122us) server_negotiation.cc:407] Connection header received
0928 14:43:23.723924 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:23.723925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:23.723936 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:23.723996 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:23.724311 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:23.724834 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:23.725558 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:23.725758 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:23.725815 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:23.728502 (+ 2687us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.66:46857: BlockingRecv error: recv got EOF from 127.9.33.66:46857 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 28 14:43:23 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:23 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:23.871256 18504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:23.866156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.67:46795)
0928 14:43:23.866304 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:23.866314 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:23.866386 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:23.866523 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:23.866523 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:23.866701 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:23.866749 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:23.866751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:23.867482 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:23.867484 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:23.868079 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:23.868081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:23.868444 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:23.868446 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:23.868515 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:23.869056 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:23.869061 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:23.871146 (+ 2085us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":93,"threads_started":1}
W20260928 14:43:23.871306 18502 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:23.866332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:46795 (local address 127.9.33.126:37447)
0928 14:43:23.866403 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:23.866405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:23.866414 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:43:23.866442 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:23.866443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:23.866458 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:23.866516 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:23.866830 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:23.867388 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:23.868164 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:23.868366 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:23.868445 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:23.871236 (+ 2791us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.67:46795: BlockingRecv error: recv got EOF from 127.9.33.67:46795 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 28 14:43:24 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:24 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:24.612110 18505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:24.607374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.65:37251)
0928 14:43:24.607533 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:24.607562 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:24.607622 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:24.607804 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:24.607804 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:24.608023 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:24.608077 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:24.608079 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:24.608731 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:24.608732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:24.609297 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:24.609298 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:24.609641 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:24.609642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:24.609685 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:24.610213 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:24.610218 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:24.612052 (+ 1834us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":89,"threads_started":1}
W20260928 14:43:24.612217 18506 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:24.607478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:37251 (local address 127.9.33.126:37447)
0928 14:43:24.607584 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:24.607586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:24.607620 (+ 34us) server_negotiation.cc:407] Connection header received
0928 14:43:24.607681 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:24.607681 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:24.607693 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:24.607785 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:24.608154 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:24.608659 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:24.609377 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:24.609571 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:24.609623 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:24.612143 (+ 2520us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.65:37251: BlockingRecv error: recv got EOF from 127.9.33.65:37251 (error 108)
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":57,"threads_started":1}
Sep 28 14:43:24 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:24 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:24.733470 18507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:24.728871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.66:52579)
0928 14:43:24.729018 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:24.729030 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:24.729102 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:24.729240 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:24.729241 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:24.729400 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:24.729452 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:24.729453 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:24.730104 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:24.730105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:24.730660 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:24.730662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:24.731009 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:24.731010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:24.731049 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:24.731520 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:24.731525 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:24.733401 (+ 1876us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":88,"threads_started":1}
W20260928 14:43:24.733605 18506 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:24.728979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:52579 (local address 127.9.33.126:37447)
0928 14:43:24.729040 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:24.729044 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:24.729080 (+ 36us) server_negotiation.cc:407] Connection header received
0928 14:43:24.729157 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:24.729158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:24.729168 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:24.729223 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:24.729522 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:24.730031 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:24.730734 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:24.730936 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:24.730993 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:24.733527 (+ 2534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.66:52579: BlockingRecv error: recv got EOF from 127.9.33.66:52579 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Sep 28 14:43:24 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:24 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:24.876571 18508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:24.871791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.67:51591)
0928 14:43:24.871919 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:24.871929 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:24.872007 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:24.872120 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:24.872120 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:24.872281 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:24.872331 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:24.872332 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:24.873027 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:24.873028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:24.873562 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:24.873563 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:24.873907 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:24.873908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:24.873947 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:24.874456 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:24.874461 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:24.876456 (+ 1995us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":69,"threads_started":1}
W20260928 14:43:24.876601 18506 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:24.871822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:51591 (local address 127.9.33.126:37447)
0928 14:43:24.871883 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:24.871884 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:24.871985 (+ 101us) server_negotiation.cc:407] Connection header received
0928 14:43:24.872051 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:24.872052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:24.872064 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:24.872115 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:24.872424 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:24.872937 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:24.873639 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:24.873839 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:24.873897 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:24.876536 (+ 2639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.67:51591: BlockingRecv error: recv got EOF from 127.9.33.67:51591 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 28 14:43:25 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:25 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:25.622651 18509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:25.612540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.65:47039)
0928 14:43:25.612741 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:25.612753 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:25.612812 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:25.613014 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:25.613015 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:25.613174 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:25.613228 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:25.613229 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:25.613884 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:25.613885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:25.614457 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:25.614459 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:25.614786 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:25.614788 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:25.614866 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:25.620582 (+ 5716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:25.620586 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:25.622584 (+ 1998us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":111,"threads_started":1}
W20260928 14:43:25.622757 18510 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:25.612614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:47039 (local address 127.9.33.126:37447)
0928 14:43:25.612727 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:25.612729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:25.612823 (+ 94us) server_negotiation.cc:407] Connection header received
0928 14:43:25.612851 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:25.612851 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:25.612862 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:25.612920 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:25.613300 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:25.613803 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:25.614526 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:25.614711 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:25.614759 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:25.622679 (+ 7920us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.65:47039: BlockingRecv error: recv got EOF from 127.9.33.65:47039 (error 108)
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":48,"threads_started":1}
Sep 28 14:43:25 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:25 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:25.738508 18511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:25.734012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.66:33259)
0928 14:43:25.734165 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:25.734175 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:25.734235 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:25.734362 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:25.734362 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:25.734535 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:25.734584 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:25.734586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:25.735253 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:25.735254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:25.735816 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:25.735818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:25.736153 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:25.736153 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:25.736192 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:25.736705 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:25.736710 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:25.738444 (+ 1734us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":93,"threads_started":1}
W20260928 14:43:25.738639 18510 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:25.734060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:33259 (local address 127.9.33.126:37447)
0928 14:43:25.734129 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:25.734132 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:25.734240 (+ 108us) server_negotiation.cc:407] Connection header received
0928 14:43:25.734283 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:25.734284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:25.734295 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:25.734347 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:25.734649 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:25.735187 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:25.735906 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:25.736090 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:25.736144 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:25.738537 (+ 2393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.66:33259: BlockingRecv error: recv got EOF from 127.9.33.66:33259 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 28 14:43:25 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:25 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:25.881510 18512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:25.877153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.67:34041)
0928 14:43:25.877321 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:25.877332 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:25.877394 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:25.877535 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:25.877536 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:25.877744 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:25.877794 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:25.877797 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:25.878469 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:25.878470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:25.878997 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:25.878999 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:25.879320 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:25.879321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:25.879361 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:25.879815 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:25.879819 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:25.881445 (+ 1626us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":109,"threads_started":1}
W20260928 14:43:25.881664 18510 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:25.877207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:34041 (local address 127.9.33.126:37447)
0928 14:43:25.877274 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:25.877276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:25.877415 (+ 139us) server_negotiation.cc:407] Connection header received
0928 14:43:25.877447 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:25.877448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:25.877462 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:25.877521 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:25.877868 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:25.878402 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:25.879075 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:25.879257 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:25.879310 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:25.881560 (+ 2250us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.67:34041: BlockingRecv error: recv got EOF from 127.9.33.67:34041 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Sep 28 14:43:26 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:26 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:26.628048 18513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:26.623166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.65:44015)
0928 14:43:26.623310 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:26.623321 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:26.623395 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:26.623550 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:26.623551 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:26.623755 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:26.623820 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:26.623822 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:26.624492 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:26.624493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:26.625091 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:26.625093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:26.625436 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:26.625437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:26.625479 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:26.626031 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:26.626036 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:26.627938 (+ 1902us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":76,"threads_started":1}
W20260928 14:43:26.628095 18514 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:26.623257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:44015 (local address 127.9.33.126:37447)
0928 14:43:26.623390 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:26.623392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:26.623400 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:43:26.623462 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:26.623463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:26.623473 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:26.623530 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:26.623888 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:26.624399 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:26.625172 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:26.625365 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:26.625417 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:26.628023 (+ 2606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.65:44015: BlockingRecv error: recv got EOF from 127.9.33.65:44015 (error 108)
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":88,"threads_started":1}
Sep 28 14:43:26 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:26 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:26.743945 18515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:26.738963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.66:45085)
0928 14:43:26.739117 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:26.739136 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:26.739218 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:26.739440 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:26.739442 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:26.739641 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:26.739691 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:26.739693 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:26.740536 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:26.740538 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:26.741087 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:26.741089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:26.741467 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:26.741468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:26.741524 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:26.741991 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:26.741995 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:26.743849 (+ 1854us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":58,"threads_started":1}
W20260928 14:43:26.743997 18514 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:26.739135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:45085 (local address 127.9.33.126:37447)
0928 14:43:26.739200 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:26.739203 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:26.739212 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:43:26.739344 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:26.739345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:26.739365 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:26.739431 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:26.739895 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:26.740448 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:26.741173 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:26.741381 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:26.741466 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:26.743930 (+ 2464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.66:45085: BlockingRecv error: recv got EOF from 127.9.33.66:45085 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 28 14:43:26 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:26 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:26.887192 18516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:26.882075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.67:59895)
0928 14:43:26.882353 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:26.882366 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:26.882454 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:26.882632 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:26.882633 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:26.882835 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:26.882884 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:26.882886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:26.883581 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:26.883582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:26.884200 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:26.884202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:26.884564 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:26.884565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:26.884627 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:26.885158 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:26.885164 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:26.887117 (+ 1953us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":188,"threads_started":1}
W20260928 14:43:26.887323 18514 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:26.882277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:59895 (local address 127.9.33.126:37447)
0928 14:43:26.882349 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:26.882351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:26.882435 (+ 84us) server_negotiation.cc:407] Connection header received
0928 14:43:26.882536 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:26.882537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:26.882551 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:26.882624 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:26.882970 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:26.883479 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:26.884287 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:26.884490 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:26.884566 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:26.887238 (+ 2672us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.67:59895: BlockingRecv error: recv got EOF from 127.9.33.67:59895 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 28 14:43:27 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:27 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:27.633075 18517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:27.628491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.65:60135)
0928 14:43:27.628641 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:27.628659 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:27.628707 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:27.628838 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:27.628839 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:27.629003 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:27.629056 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:27.629057 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:27.629742 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:27.629743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:27.630293 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:27.630295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:27.630637 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:27.630638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:27.630680 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:27.631198 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:27.631202 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:27.633017 (+ 1815us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":85,"threads_started":1}
W20260928 14:43:27.633247 18518 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:27.628565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:60135 (local address 127.9.33.126:37447)
0928 14:43:27.628716 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:27.628717 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:27.628725 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:43:27.628751 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:27.628751 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:27.628762 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:27.628824 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:27.629131 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:27.629669 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:27.630366 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:27.630574 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:27.630625 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:27.633147 (+ 2522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.65:60135: BlockingRecv error: recv got EOF from 127.9.33.65:60135 (error 108)
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":94,"threads_started":1}
Sep 28 14:43:27 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:27 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:27.749480 18519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:27.744539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.66:47607)
0928 14:43:27.744698 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:27.744711 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:27.744786 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:27.744972 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:27.744974 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:27.745229 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:27.745279 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:27.745281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:27.745989 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:27.745990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:27.746593 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:27.746596 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:27.746941 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:27.746942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:27.746984 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:27.747408 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:27.747413 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:27.749418 (+ 2005us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":89,"threads_started":1}
W20260928 14:43:27.749648 18518 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:27.744588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:47607 (local address 127.9.33.126:37447)
0928 14:43:27.744641 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:27.744644 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:27.744765 (+ 121us) server_negotiation.cc:407] Connection header received
0928 14:43:27.744878 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:27.744879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:27.744895 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:27.744951 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:27.745382 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:27.745916 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:27.746679 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:27.746874 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:27.746933 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:27.749562 (+ 2629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.66:47607: BlockingRecv error: recv got EOF from 127.9.33.66:47607 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Sep 28 14:43:27 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:27 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:27.892139 18520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:27.887769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.67:41221)
0928 14:43:27.887974 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:27.887984 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:27.888059 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:27.888182 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:27.888182 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:27.888340 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:27.888389 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:27.888391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:27.889033 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:27.889033 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:27.889546 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:27.889547 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:27.889849 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:27.889849 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:27.889894 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:27.890336 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:27.890341 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:27.892079 (+ 1738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":142,"threads_started":1}
W20260928 14:43:27.892297 18518 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:27.887789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:41221 (local address 127.9.33.126:37447)
0928 14:43:27.887840 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:27.887842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:27.888065 (+ 223us) server_negotiation.cc:407] Connection header received
0928 14:43:27.888107 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:27.888108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:27.888121 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:27.888160 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:27.888450 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:27.888957 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:27.889614 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:27.889787 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:27.889837 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:27.892211 (+ 2374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.67:41221: BlockingRecv error: recv got EOF from 127.9.33.67:41221 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Sep 28 14:43:28 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:28 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:28.639107 18521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:28.633625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.65:60627)
0928 14:43:28.633763 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:28.633776 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:28.633858 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:28.634156 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:28.634157 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:28.634318 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:28.634371 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:28.634372 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:28.635210 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:28.635211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:28.635862 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:28.635864 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:28.636302 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:28.636304 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:28.636362 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:28.636845 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:28.636850 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:28.639006 (+ 2156us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":68,"threads_started":1}
W20260928 14:43:28.639216 18522 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:28.633735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:60627 (local address 127.9.33.126:37447)
0928 14:43:28.633954 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:28.633957 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:28.633968 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:43:28.634002 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:28.634003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:28.634016 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:28.634117 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:28.634447 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:28.635127 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:28.635979 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:28.636186 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:28.636251 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:28.639128 (+ 2877us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.65:60627: BlockingRecv error: recv got EOF from 127.9.33.65:60627 (error 108)
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":93,"threads_started":1}
Sep 28 14:43:28 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:28 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:28.754379 18523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:28.749960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.66:52187)
0928 14:43:28.750088 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:28.750099 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:28.750152 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:28.750271 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:28.750272 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:28.750438 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:28.750487 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:28.750489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:28.751167 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:28.751168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:28.751694 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:28.751696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:28.752116 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:28.752117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:28.752164 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:28.752612 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:28.752616 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:28.754317 (+ 1701us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":60,"threads_started":1}
W20260928 14:43:28.754529 18522 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:28.750032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:52187 (local address 127.9.33.126:37447)
0928 14:43:28.750134 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:28.750136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:28.750146 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:43:28.750206 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:28.750207 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:28.750218 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:28.750265 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:28.750581 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:28.751090 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:28.751835 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:28.752049 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:28.752107 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:28.754434 (+ 2327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.66:52187: BlockingRecv error: recv got EOF from 127.9.33.66:52187 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Sep 28 14:43:28 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:28 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:28.897199 18524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:28.892644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.67:52373)
0928 14:43:28.892764 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:28.892776 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:28.892843 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:28.893007 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:28.893008 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:28.893190 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:28.893236 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:28.893238 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:28.893966 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:28.893970 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:28.894494 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:28.894495 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:28.894886 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:28.894887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:28.894957 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:28.895402 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:28.895406 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:28.897141 (+ 1735us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":68,"threads_started":1}
W20260928 14:43:28.897320 18522 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:28.892710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:52373 (local address 127.9.33.126:37447)
0928 14:43:28.892758 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:28.892760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:28.892823 (+ 63us) server_negotiation.cc:407] Connection header received
0928 14:43:28.892912 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:28.892913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:28.892927 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:28.893000 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:28.893318 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:28.893840 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:28.894591 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:28.894800 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:28.894882 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:28.897237 (+ 2355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.67:52373: BlockingRecv error: recv got EOF from 127.9.33.67:52373 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Sep 28 14:43:29 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:29 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:29.644829 18525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:29.639656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.65:58887)
0928 14:43:29.639806 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:29.639821 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:29.639898 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:29.640100 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:29.640100 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:29.640263 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:29.640316 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:29.640318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:29.641161 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:29.641162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:29.641700 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:29.641702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:29.642095 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:29.642096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:29.642141 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:29.642663 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:29.642668 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:29.644762 (+ 2094us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":120,"threads_started":1}
W20260928 14:43:29.645089 18526 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:29.639830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:58887 (local address 127.9.33.126:37447)
0928 14:43:29.639960 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:29.639962 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:29.639972 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:43:29.640003 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:29.640004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:29.640016 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:29.640077 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:29.640425 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:29.641087 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:29.641786 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:29.642025 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:29.642082 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:29.645016 (+ 2934us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.65:58887: BlockingRecv error: recv got EOF from 127.9.33.65:58887 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":62,"threads_started":1}
Sep 28 14:43:29 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:29 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:29.760720 18527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:29.755038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.66:50113)
0928 14:43:29.755191 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:29.755202 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:29.755258 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:29.755449 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:29.755451 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:29.755612 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:29.755664 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:29.755666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:29.756777 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:29.756778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:29.757566 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:29.757569 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:29.758011 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:29.758012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:29.758078 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:29.758605 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:29.758618 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:29.760661 (+ 2043us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":105,"threads_started":1}
W20260928 14:43:29.760886 18526 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:29.755171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:50113 (local address 127.9.33.126:37447)
0928 14:43:29.755233 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:29.755235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:29.755246 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:43:29.755333 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:29.755334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:29.755346 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:29.755402 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:29.755773 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:29.756674 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:29.757695 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:29.757896 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:29.757969 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:29.760806 (+ 2837us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.66:50113: BlockingRecv error: recv got EOF from 127.9.33.66:50113 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Sep 28 14:43:29 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:29 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:29.902372 18528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:29.897747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.67:46183)
0928 14:43:29.897958 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:29.897969 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:29.898038 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:29.898177 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:29.898178 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:29.898338 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:29.898388 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:29.898390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:29.899038 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:29.899040 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:29.899573 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:29.899574 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:29.899946 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:29.899947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:29.899999 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:29.900459 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:29.900469 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:29.902301 (+ 1832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":70,"threads_started":1}
W20260928 14:43:29.902518 18526 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:29.897842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:46183 (local address 127.9.33.126:37447)
0928 14:43:29.897899 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:29.897901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:29.898040 (+ 139us) server_negotiation.cc:407] Connection header received
0928 14:43:29.898084 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:29.898085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:29.898098 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:29.898147 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:29.898454 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:29.898944 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:29.899660 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:29.899871 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:29.899927 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:29.902400 (+ 2473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.67:46183: BlockingRecv error: recv got EOF from 127.9.33.67:46183 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Sep 28 14:43:30 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:30 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:30.654667 18529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:30.645306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.65:55851)
0928 14:43:30.645476 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:30.645493 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:30.645549 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:30.645739 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:30.645740 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:30.645910 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:30.645963 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:30.645964 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:30.646649 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:30.646650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:30.647251 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:30.647253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:30.647672 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:30.647674 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:30.647814 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:30.652411 (+ 4597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:30.652424 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:30.654570 (+ 2146us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":116,"threads_started":1}
W20260928 14:43:30.654738 18530 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:30.645369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:55851 (local address 127.9.33.126:37447)
0928 14:43:30.645500 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:30.645502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:30.645533 (+ 31us) server_negotiation.cc:407] Connection header received
0928 14:43:30.645638 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:30.645639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:30.645653 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:30.645724 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:30.646030 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:30.646570 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:30.647318 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:30.647525 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:30.647583 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:30.654662 (+ 7079us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.65:55851: BlockingRecv error: recv got EOF from 127.9.33.65:55851 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":71,"threads_started":1}
Sep 28 14:43:30 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:30 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:30.766146 18531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:30.761224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.66:39457)
0928 14:43:30.761390 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:30.761402 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:30.761467 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:30.761605 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:30.761606 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:30.761774 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:30.761822 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:30.761824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:30.762579 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:30.762581 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:30.763384 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:30.763387 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:30.763778 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:30.763779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:30.763838 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:30.764310 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:30.764315 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:30.766088 (+ 1773us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":110,"threads_started":1}
W20260928 14:43:30.766276 18530 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:30.761318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:39457 (local address 127.9.33.126:37447)
0928 14:43:30.761388 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:30.761389 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:30.761469 (+ 80us) server_negotiation.cc:407] Connection header received
0928 14:43:30.761516 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:30.761517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:30.761528 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:30.761583 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:30.761913 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:30.762472 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:30.763464 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:30.763656 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:30.763711 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:30.766191 (+ 2480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.66:39457: BlockingRecv error: recv got EOF from 127.9.33.66:39457 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 28 14:43:30 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:30 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:30.908192 18532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:30.902820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.67:53819)
0928 14:43:30.902990 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:30.903005 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:30.903075 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:30.903260 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:30.903262 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:30.903476 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:30.903543 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:30.903545 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:30.904713 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:30.904714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:30.905391 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:30.905393 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:30.905716 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:30.905717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:30.905760 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:30.906224 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:30.906228 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:30.908124 (+ 1896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":67,"threads_started":1}
W20260928 14:43:30.908336 18530 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:30.902950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:53819 (local address 127.9.33.126:37447)
0928 14:43:30.903005 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:30.903007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:30.903059 (+ 52us) server_negotiation.cc:407] Connection header received
0928 14:43:30.903130 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:30.903131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:30.903144 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:30.903184 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:30.903772 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:30.904633 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:30.905470 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:30.905653 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:30.905705 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:30.908252 (+ 2547us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.67:53819: BlockingRecv error: recv got EOF from 127.9.33.67:53819 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Sep 28 14:43:31 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:31 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:31.659967 18533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:31.655186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.65:43967)
0928 14:43:31.655339 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:31.655350 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:31.655410 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:31.655546 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:31.655546 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:31.655734 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:31.655789 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:31.655790 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:31.656571 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:31.656573 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:31.657159 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:31.657162 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:31.657514 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:31.657515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:31.657558 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:31.658067 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:31.658071 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:31.659905 (+ 1834us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":83,"threads_started":1}
W20260928 14:43:31.660138 18534 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:31.655282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:43967 (local address 127.9.33.126:37447)
0928 14:43:31.655400 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:31.655401 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:31.655411 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:43:31.655458 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:31.655459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:31.655469 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:31.655526 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:31.655895 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:31.656483 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:31.657239 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:31.657448 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:31.657502 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:31.660048 (+ 2546us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.65:43967: BlockingRecv error: recv got EOF from 127.9.33.65:43967 (error 108)
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":51,"threads_started":1}
Sep 28 14:43:31 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:31 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:31.772043 18535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:31.766790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.66:48271)
0928 14:43:31.766986 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:31.766997 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:31.767063 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:31.767205 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:31.767206 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:31.767373 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:31.767423 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:31.767424 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:31.768111 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:31.768112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:31.768716 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:31.768718 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:31.769114 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:31.769115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:31.769170 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:31.769772 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:31.769777 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:31.771982 (+ 2205us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":83,"threads_started":1}
W20260928 14:43:31.772212 18534 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:31.766916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:48271 (local address 127.9.33.126:37447)
0928 14:43:31.766985 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:31.766987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:31.767059 (+ 72us) server_negotiation.cc:407] Connection header received
0928 14:43:31.767115 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:31.767115 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:31.767135 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:31.767186 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:31.767481 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:31.768015 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:31.768818 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:31.769023 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:31.769081 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:31.772123 (+ 3042us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.66:48271: BlockingRecv error: recv got EOF from 127.9.33.66:48271 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 28 14:43:31 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:31 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:31.913501 18536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:31.908662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.67:41479)
0928 14:43:31.908819 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:31.908831 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:31.908892 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:31.909193 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:31.909193 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:31.909356 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:31.909407 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:31.909409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:31.910126 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:31.910127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:31.910672 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:31.910674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:31.911067 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:31.911068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:31.911113 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:31.911578 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:31.911582 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:31.913420 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":61,"threads_started":1}
W20260928 14:43:31.913586 18534 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:31.908837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:41479 (local address 127.9.33.126:37447)
0928 14:43:31.909054 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:31.909057 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:31.909068 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:43:31.909102 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:31.909103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:31.909117 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:31.909171 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:31.909490 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:31.910051 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:31.910759 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:31.910994 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:31.911058 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:31.913533 (+ 2475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.67:41479: BlockingRecv error: recv got EOF from 127.9.33.67:41479 (error 108)
Metrics: {"server-negotiator.queue_time_us":190}
Sep 28 14:43:32 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:32 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:32.665680 18537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:32.660646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.65:38533)
0928 14:43:32.660922 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:32.660957 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:32.661030 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:32.661235 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:32.661237 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:32.661493 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:32.661546 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:32.661548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:32.662283 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:32.662284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:32.662845 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:32.662846 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:32.663212 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:32.663213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:32.663260 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:32.663814 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:32.663819 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:32.665621 (+ 1802us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":87,"threads_started":1}
W20260928 14:43:32.665890 18538 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:32.660698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:38533 (local address 127.9.33.126:37447)
0928 14:43:32.660912 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:32.660915 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:32.661075 (+ 160us) server_negotiation.cc:407] Connection header received
0928 14:43:32.661105 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:32.661106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:32.661121 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:32.661183 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:32.661631 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:32.662202 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:32.662934 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:32.663139 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:32.663197 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:32.665765 (+ 2568us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.65:38533: BlockingRecv error: recv got EOF from 127.9.33.65:38533 (error 108)
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":88,"threads_started":1}
Sep 28 14:43:32 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:32 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:32.777318 18539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:32.772569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.66:44955)
0928 14:43:32.772708 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:32.772721 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:32.772786 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:32.772946 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:32.772953 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:32.773142 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:32.773187 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:32.773189 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:32.773890 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:32.773891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:32.774435 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:32.774437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:32.774824 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:32.774825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:32.774890 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:32.775407 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:32.775412 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:32.777244 (+ 1832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":86,"threads_started":1}
W20260928 14:43:32.777439 18538 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:32.772703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:44955 (local address 127.9.33.126:37447)
0928 14:43:32.772760 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:32.772762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:32.772772 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:43:32.772847 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:32.772847 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:32.772861 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:32.772941 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:32.773272 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:32.773803 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:32.774533 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:32.774735 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:32.774826 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:32.777371 (+ 2545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.66:44955: BlockingRecv error: recv got EOF from 127.9.33.66:44955 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Sep 28 14:43:32 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:32 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:32.918992 18540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:32.914066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.67:48757)
0928 14:43:32.914222 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:32.914234 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:32.914298 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:32.914422 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:32.914423 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:32.914590 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:32.914639 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:32.914641 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:32.915301 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:32.915302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:32.915947 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:32.915950 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:32.916306 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:32.916307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:32.916378 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:32.916913 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:32.916918 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:32.918903 (+ 1985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":98,"threads_started":1}
W20260928 14:43:32.919093 18538 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:32.914243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:48757 (local address 127.9.33.126:37447)
0928 14:43:32.914316 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:32.914318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:32.914327 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:43:32.914350 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:32.914350 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:32.914362 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:32.914416 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:32.914707 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:32.915220 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:32.916040 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:32.916230 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:32.916287 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:32.918994 (+ 2707us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.67:48757: BlockingRecv error: recv got EOF from 127.9.33.67:48757 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 28 14:43:33 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:33 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:33.671257 18541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:33.666256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.65:58611)
0928 14:43:33.666400 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:33.666413 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:33.666475 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:33.666697 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:33.666699 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:33.666912 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:33.666963 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:33.666965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:33.667604 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:33.667605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:33.668242 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:33.668245 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:33.668624 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:33.668625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:33.668692 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:33.669166 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:33.669171 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:33.671193 (+ 2022us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":66,"threads_started":1}
W20260928 14:43:33.671371 18542 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:33.666331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:58611 (local address 127.9.33.126:37447)
0928 14:43:33.666491 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:33.666493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:33.666502 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:43:33.666542 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:33.666543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:33.666559 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:33.666650 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:33.667037 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:33.667535 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:33.668326 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:33.668518 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:33.668571 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:33.671295 (+ 2724us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.65:58611: BlockingRecv error: recv got EOF from 127.9.33.65:58611 (error 108)
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":103,"threads_started":1}
Sep 28 14:43:33 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:33 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:33.782392 18543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:33.777950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.66:59925)
0928 14:43:33.778110 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:33.778124 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:33.778189 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:33.778324 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:33.778325 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:33.778509 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:33.778557 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:33.778558 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:33.779212 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:33.779213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:33.779781 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:33.779783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:33.780104 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:33.780105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:33.780147 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:33.780604 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:33.780608 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:33.782320 (+ 1712us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":109,"threads_started":1}
W20260928 14:43:33.782509 18542 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:33.778038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:59925 (local address 127.9.33.126:37447)
0928 14:43:33.778098 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:33.778100 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:33.778188 (+ 88us) server_negotiation.cc:407] Connection header received
0928 14:43:33.778235 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:33.778235 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:33.778247 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:33.778301 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:33.778622 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:33.779136 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:33.779847 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:33.780039 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:33.780090 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:33.782432 (+ 2342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.66:59925: BlockingRecv error: recv got EOF from 127.9.33.66:59925 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Sep 28 14:43:33 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:33 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:33.924211 18544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:33.919484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.67:38841)
0928 14:43:33.919635 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:33.919647 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:33.919702 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:33.919900 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:33.919901 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:33.920071 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:33.920119 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:33.920120 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:33.920757 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:33.920758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:33.921362 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:33.921365 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:33.921724 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:33.921725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:33.921784 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:33.922276 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:33.922282 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:33.924137 (+ 1855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":97,"threads_started":1}
W20260928 14:43:33.924319 18542 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:33.919604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:38841 (local address 127.9.33.126:37447)
0928 14:43:33.919682 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:33.919684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:33.919693 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:43:33.919799 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:33.919800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:33.919815 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:33.919883 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:33.920191 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:33.920680 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:33.921436 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:33.921623 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:33.921673 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:33.924257 (+ 2584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.67:38841: BlockingRecv error: recv got EOF from 127.9.33.67:38841 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 28 14:43:34 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:34 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:34.676542 18545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:34.671899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.65:48913)
0928 14:43:34.672067 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:34.672080 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:34.672150 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:34.672359 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:34.672360 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:34.672529 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:34.672578 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:34.672579 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:34.673234 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:34.673235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:34.673760 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:34.673761 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:34.674094 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:34.674095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:34.674133 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:34.674561 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:34.674566 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:34.676478 (+ 1912us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":91,"threads_started":1}
W20260928 14:43:34.676659 18546 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:34.671964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:48913 (local address 127.9.33.126:37447)
0928 14:43:34.672122 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:34.672124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:34.672136 (+ 12us) server_negotiation.cc:407] Connection header received
0928 14:43:34.672247 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:34.672248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:34.672264 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:34.672326 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:34.672644 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:34.673167 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:34.673827 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:34.674031 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:34.674082 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:34.676577 (+ 2495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.65:48913: BlockingRecv error: recv got EOF from 127.9.33.65:48913 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":85,"threads_started":1}
Sep 28 14:43:34 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:34 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:34.787518 18547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:34.782945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.66:43713)
0928 14:43:34.783116 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:34.783127 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:34.783184 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:34.783314 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:34.783315 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:34.783506 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:34.783555 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:34.783556 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:34.784263 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:34.784264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:34.784808 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:34.784810 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:34.785163 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:34.785164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:34.785210 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:34.785660 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:34.785665 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:34.787455 (+ 1790us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":99,"threads_started":1}
W20260928 14:43:34.787627 18546 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:34.783047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:43713 (local address 127.9.33.126:37447)
0928 14:43:34.783112 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:34.783114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:34.783191 (+ 77us) server_negotiation.cc:407] Connection header received
0928 14:43:34.783232 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:34.783232 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:34.783244 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:34.783293 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:34.783619 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:34.784189 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:34.784888 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:34.785094 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:34.785147 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:34.787554 (+ 2407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.66:43713: BlockingRecv error: recv got EOF from 127.9.33.66:43713 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Sep 28 14:43:34 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:34 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:34.929492 18548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:34.924803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.67:60177)
0928 14:43:34.924951 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:34.924966 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:34.925022 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:34.925133 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:34.925134 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:34.925288 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:34.925337 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:34.925338 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:34.925956 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:34.925957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:34.926488 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:34.926489 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:34.926819 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:34.926819 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:34.926861 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:34.927283 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:34.927287 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:34.929387 (+ 2100us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":82,"threads_started":1}
W20260928 14:43:34.929574 18546 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:34.924855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:60177 (local address 127.9.33.126:37447)
0928 14:43:34.924920 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:34.924922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:34.925026 (+ 104us) server_negotiation.cc:407] Connection header received
0928 14:43:34.925065 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:34.925065 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:34.925078 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:34.925123 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:34.925392 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:34.925888 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:34.926572 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:34.926754 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:34.926806 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:34.929503 (+ 2697us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.67:60177: BlockingRecv error: recv got EOF from 127.9.33.67:60177 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 28 14:43:35 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:35 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:35.689013 18549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:35.677212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.65:49047)
0928 14:43:35.677389 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:35.677402 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:35.677469 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:35.677668 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:35.677669 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:35.677844 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:35.677908 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:35.677909 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:35.678661 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:35.678662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:35.679239 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:35.679241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:35.679620 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:35.679621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:35.679670 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:35.686824 (+ 7154us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:35.686830 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:35.688951 (+ 2121us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":70,"threads_started":1}
W20260928 14:43:35.689181 18550 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:35.677261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:49047 (local address 127.9.33.126:37447)
0928 14:43:35.677451 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:35.677452 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:35.677464 (+ 12us) server_negotiation.cc:407] Connection header received
0928 14:43:35.677536 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:35.677537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:35.677549 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:35.677603 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:35.677977 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:35.678569 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:35.679342 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:35.679543 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:35.679599 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:35.689078 (+ 9479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.65:49047: BlockingRecv error: recv got EOF from 127.9.33.65:49047 (error 108)
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":103,"threads_started":1}
Sep 28 14:43:35 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:35 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:35.792892 18551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:35.788053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.66:37689)
0928 14:43:35.788216 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:35.788226 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:35.788295 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:35.788444 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:35.788445 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:35.788634 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:35.788694 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:35.788696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:35.789443 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:35.789445 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:35.790003 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:35.790005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:35.790390 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:35.790391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:35.790435 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:35.790878 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:35.790882 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:35.792784 (+ 1902us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":99,"threads_started":1}
W20260928 14:43:35.792937 18550 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:35.788167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:37689 (local address 127.9.33.126:37447)
0928 14:43:35.788222 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:35.788224 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:35.788279 (+ 55us) server_negotiation.cc:407] Connection header received
0928 14:43:35.788350 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:35.788351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:35.788362 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:35.788419 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:35.788801 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:35.789342 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:35.790115 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:35.790315 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:35.790380 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:35.792883 (+ 2503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.66:37689: BlockingRecv error: recv got EOF from 127.9.33.66:37689 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Sep 28 14:43:35 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:35 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.67@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:35.936154 18552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:35.930184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.67:57715)
0928 14:43:35.930361 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:35.930375 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:35.930472 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:35.930647 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:35.930648 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:35.930830 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:35.930877 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:35.930878 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:35.931698 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:35.931700 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:35.932592 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:35.932595 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:35.933117 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:35.933118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:35.933199 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:35.933833 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:35.933839 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:35.936095 (+ 2256us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":99,"threads_started":1}
W20260928 14:43:35.936257 18550 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:35.930291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:57715 (local address 127.9.33.126:37447)
0928 14:43:35.930365 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:35.930368 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:35.930450 (+ 82us) server_negotiation.cc:407] Connection header received
0928 14:43:35.930559 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:35.930560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:35.930574 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:35.930631 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:35.930989 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:35.931585 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:35.932699 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:35.933019 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:35.933096 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:35.936188 (+ 3092us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.67:57715: BlockingRecv error: recv got EOF from 127.9.33.67:57715 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 28 14:43:36 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:36 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.65@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:36.694159 18553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:36.689563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.65:37057)
0928 14:43:36.689710 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:36.689722 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:36.689782 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:36.689985 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:36.689986 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:36.690146 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:36.690198 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:36.690200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:36.690851 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:36.690852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:36.691402 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:36.691404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:36.691820 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:36.691821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:36.691864 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:36.692322 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:36.692326 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:36.694100 (+ 1774us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":87,"threads_started":1}
W20260928 14:43:36.694353 18554 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:36.689652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:37057 (local address 127.9.33.126:37447)
0928 14:43:36.689794 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:36.689796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:36.689812 (+ 16us) server_negotiation.cc:407] Connection header received
0928 14:43:36.689835 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:36.689836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:36.689888 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:36.689953 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:36.690268 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:36.690775 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:36.691493 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:36.691690 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:36.691818 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:36.694224 (+ 2406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.65:37057: BlockingRecv error: recv got EOF from 127.9.33.65:37057 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":84,"threads_started":1}
Sep 28 14:43:36 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
Sep 28 14:43:36 dist-test-slave-pv0w krb5kdc[17495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.33.66@KRBTEST.COM for oryx/127.9.33.126@KRBTEST.COM, Server not found in Kerberos database
I20260928 14:43:36.798859 18555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:36.793632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37447 (local address 127.9.33.66:51583)
0928 14:43:36.793765 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:36.793779 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:36.794053 (+ 274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:36.794260 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:36.794262 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:36.794452 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:36.794521 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:36.794523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:36.795252 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:36.795253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:36.795875 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:36.795877 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:36.796266 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:36.796268 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:36.796327 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:36.796824 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:36.796828 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:36.798763 (+ 1935us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.33.126:37447: Server oryx/127.9.33.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":75,"threads_started":1}
W20260928 14:43:36.798923 18554 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:36.793748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:51583 (local address 127.9.33.126:37447)
0928 14:43:36.793828 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:36.793831 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:36.794044 (+ 213us) server_negotiation.cc:407] Connection header received
0928 14:43:36.794156 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:36.794157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:36.794171 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:36.794250 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:36.794614 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:36.795152 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:36.795962 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:36.796175 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:36.796266 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:36.798857 (+ 2591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.33.66:51583: BlockingRecv error: recv got EOF from 127.9.33.66:51583 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 14:43:36.851537 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 18078
I20260928 14:43:36.856482 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 18228
I20260928 14:43:36.860669 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 18364
I20260928 14:43:36.864893 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 18008
2026-09-28T14:43:36Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17550 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-28T14:43:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:43:36Z Disabled control of system clock
I20260928 14:43:36.892537 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:33819
--webserver_interface=127.9.33.126
--webserver_port=0
--builtin_ntp_servers=127.9.33.84:32973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:33819
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260928 14:43:36.974171 18559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:36.974407 18559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:36.974428 18559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:36.975630 18559 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:43:36.975672 18559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:36.975685 18559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:36.975698 18559 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:43:36.975710 18559 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:43:36.977180 18559 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:32973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:33819
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:33819
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:36.977427 18559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:36.977690 18559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 14:43:36.983291 18559 server_base.cc:1034] running on GCE node
W20260928 14:43:36.983239 18567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:36.983243 18565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:36.983239 18564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:36.983889 18559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:36.984201 18559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:36.985399 18559 hybrid_clock.cc:648] HybridClock initialized: now 1790606616985385 us; error 30 us; skew 500 ppm
I20260928 14:43:36.986716 18559 webserver.cc:460] Webserver started at http://127.9.33.126:43989/ using document root <none> and password file <none>
I20260928 14:43:36.987010 18559 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:36.987087 18559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:36.987248 18559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:36.988430 18559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "48c5ba6c2876496e9e9543f0f1725c54"
format_stamp: "Formatted at 2026-09-28 14:43:36 on dist-test-slave-pv0w"
server_key: "94efb6788543f173d9621bf3d57f4491"
server_key_iv: "eed537bf42771cedba11cff3ca8ec10a"
server_key_version: "encryptionkey@0"
I20260928 14:43:36.988796 18559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "48c5ba6c2876496e9e9543f0f1725c54"
format_stamp: "Formatted at 2026-09-28 14:43:36 on dist-test-slave-pv0w"
server_key: "94efb6788543f173d9621bf3d57f4491"
server_key_iv: "eed537bf42771cedba11cff3ca8ec10a"
server_key_version: "encryptionkey@0"
I20260928 14:43:36.990190 18559 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260928 14:43:36.991129 18573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:36.991348 18559 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:36.991474 18559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "48c5ba6c2876496e9e9543f0f1725c54"
format_stamp: "Formatted at 2026-09-28 14:43:36 on dist-test-slave-pv0w"
server_key: "94efb6788543f173d9621bf3d57f4491"
server_key_iv: "eed537bf42771cedba11cff3ca8ec10a"
server_key_version: "encryptionkey@0"
I20260928 14:43:36.991555 18559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:37.006419 18559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:37.006771 18559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:37.006932 18559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:37.011101 18559 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:33819
I20260928 14:43:37.011138 18625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:33819 every 8 connection(s)
I20260928 14:43:37.011538 18559 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:43:37.012614 18626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:37.014954 18626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48c5ba6c2876496e9e9543f0f1725c54: Bootstrap starting.
I20260928 14:43:37.015551 18626 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48c5ba6c2876496e9e9543f0f1725c54: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:37.015918 18626 log.cc:824] T 00000000000000000000000000000000 P 48c5ba6c2876496e9e9543f0f1725c54: Log is configured to *not* fsync() on all Append() calls
I20260928 14:43:37.016589 18626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48c5ba6c2876496e9e9543f0f1725c54: No bootstrap required, opened a new log
I20260928 14:43:37.016619 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 18559
I20260928 14:43:37.016762 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:43:37.016957 9349 external_mini_cluster.cc:1428] Setting key bec59c52af69db59f34831d9ff556ebb
I20260928 14:43:37.018877 18626 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48c5ba6c2876496e9e9543f0f1725c54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c5ba6c2876496e9e9543f0f1725c54" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 33819 } }
I20260928 14:43:37.019060 18626 raft_consensus.cc:374] T 00000000000000000000000000000000 P 48c5ba6c2876496e9e9543f0f1725c54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:37.019097 18626 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48c5ba6c2876496e9e9543f0f1725c54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48c5ba6c2876496e9e9543f0f1725c54, State: Initialized, Role: FOLLOWER
I20260928 14:43:37.019268 18626 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48c5ba6c2876496e9e9543f0f1725c54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c5ba6c2876496e9e9543f0f1725c54" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 33819 } }
I20260928 14:43:37.019389 18626 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48c5ba6c2876496e9e9543f0f1725c54 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:43:37.019423 18626 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48c5ba6c2876496e9e9543f0f1725c54 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:43:37.019474 18626 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48c5ba6c2876496e9e9543f0f1725c54 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:37.020267 18626 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48c5ba6c2876496e9e9543f0f1725c54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c5ba6c2876496e9e9543f0f1725c54" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 33819 } }
I20260928 14:43:37.020377 18626 leader_election.cc:302] T 00000000000000000000000000000000 P 48c5ba6c2876496e9e9543f0f1725c54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48c5ba6c2876496e9e9543f0f1725c54; no voters:
I20260928 14:43:37.020524 18626 leader_election.cc:288] T 00000000000000000000000000000000 P 48c5ba6c2876496e9e9543f0f1725c54 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:43:37.020582 18631 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48c5ba6c2876496e9e9543f0f1725c54 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:43:37.020726 18631 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48c5ba6c2876496e9e9543f0f1725c54 [term 1 LEADER]: Becoming Leader. State: Replica: 48c5ba6c2876496e9e9543f0f1725c54, State: Running, Role: LEADER
I20260928 14:43:37.020759 18626 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48c5ba6c2876496e9e9543f0f1725c54 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:43:37.022128 18631 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48c5ba6c2876496e9e9543f0f1725c54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c5ba6c2876496e9e9543f0f1725c54" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 33819 } }
I20260928 14:43:37.022724 18633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48c5ba6c2876496e9e9543f0f1725c54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48c5ba6c2876496e9e9543f0f1725c54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c5ba6c2876496e9e9543f0f1725c54" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 33819 } } }
I20260928 14:43:37.022922 18633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48c5ba6c2876496e9e9543f0f1725c54 [sys.catalog]: This master's current role is: LEADER
I20260928 14:43:37.023151 18632 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48c5ba6c2876496e9e9543f0f1725c54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48c5ba6c2876496e9e9543f0f1725c54. Latest consensus state: current_term: 1 leader_uuid: "48c5ba6c2876496e9e9543f0f1725c54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48c5ba6c2876496e9e9543f0f1725c54" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 33819 } } }
I20260928 14:43:37.023238 18632 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48c5ba6c2876496e9e9543f0f1725c54 [sys.catalog]: This master's current role is: LEADER
W20260928 14:43:37.023423 18645 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 48c5ba6c2876496e9e9543f0f1725c54: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:43:37.023483 18645 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:43:37.023547 18647 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:43:37.023742 18647 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:43:37.024497 18630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:37.017517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55634 (local address 127.9.33.126:33819)
0928 14:43:37.017756 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:37.017765 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:37.017800 (+ 35us) server_negotiation.cc:407] Connection header received
0928 14:43:37.018839 (+ 1039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:37.018849 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:37.018930 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:37.019086 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:37.019361 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:37.021125 (+ 1764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:37.022204 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:37.022813 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:37.022923 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:37.023862 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:37.023867 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:37.023867 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:43:37.024050 (+ 183us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:37.024083 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:43:37.024094 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:37.024170 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:37.024246 (+ 76us) server_negotiation.cc:299] Negotiation successful
0928 14:43:37.024327 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":97,"threads_started":1}
I20260928 14:43:37.028982 18647 catalog_manager.cc:1347] Generated new cluster ID: db32007a67d644e88df888312278bff3
I20260928 14:43:37.029043 18647 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:43:37.048905 18647 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:43:37.049467 18647 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:43:37.051222 18647 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48c5ba6c2876496e9e9543f0f1725c54: Generated new TSK 0
I20260928 14:43:37.051397 18647 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:43:37.058120 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.65:0
--local_ip_for_outbound_sockets=127.9.33.65
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:33819
--builtin_ntp_servers=127.9.33.84:32973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260928 14:43:37.153751 18650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:37.153980 18650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:37.154004 18650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:37.155279 18650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:37.155326 18650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:37.155354 18650 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.65
I20260928 14:43:37.156912 18650 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:32973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:33819
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.65
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:37.157179 18650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:37.157524 18650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:37.163830 18655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:37.163838 18656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:37.164083 18650 server_base.cc:1034] running on GCE node
W20260928 14:43:37.163834 18658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:37.164499 18650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:37.164813 18650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:37.165984 18650 hybrid_clock.cc:648] HybridClock initialized: now 1790606617165966 us; error 34 us; skew 500 ppm
I20260928 14:43:37.167303 18650 webserver.cc:460] Webserver started at http://127.9.33.65:40523/ using document root <none> and password file <none>
I20260928 14:43:37.167577 18650 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:37.167624 18650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:37.167788 18650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:37.168907 18650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/data/instance:
uuid: "72c8b456c42844b5aad1248091d6f209"
format_stamp: "Formatted at 2026-09-28 14:43:37 on dist-test-slave-pv0w"
server_key: "0506921c9fd6be0471f182a16ed9d6d9"
server_key_iv: "8acbc3d2d8a4cb4b427342eaad9ccf04"
server_key_version: "encryptionkey@0"
I20260928 14:43:37.169306 18650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance:
uuid: "72c8b456c42844b5aad1248091d6f209"
format_stamp: "Formatted at 2026-09-28 14:43:37 on dist-test-slave-pv0w"
server_key: "0506921c9fd6be0471f182a16ed9d6d9"
server_key_iv: "8acbc3d2d8a4cb4b427342eaad9ccf04"
server_key_version: "encryptionkey@0"
I20260928 14:43:37.170867 18650 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:43:37.171809 18664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:37.172046 18650 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:43:37.172178 18650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/wal
uuid: "72c8b456c42844b5aad1248091d6f209"
format_stamp: "Formatted at 2026-09-28 14:43:37 on dist-test-slave-pv0w"
server_key: "0506921c9fd6be0471f182a16ed9d6d9"
server_key_iv: "8acbc3d2d8a4cb4b427342eaad9ccf04"
server_key_version: "encryptionkey@0"
I20260928 14:43:37.172274 18650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:37.183569 18650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:37.183993 18650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:37.184151 18650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:37.184379 18650 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:43:37.184737 18650 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:43:37.184794 18650 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:37.184836 18650 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:43:37.184865 18650 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:37.190800 18650 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.65:40199
I20260928 14:43:37.190848 18777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.65:40199 every 8 connection(s)
I20260928 14:43:37.191211 18650 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
I20260928 14:43:37.193991 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 18650
I20260928 14:43:37.194073 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance
I20260928 14:43:37.194324 9349 external_mini_cluster.cc:1428] Setting key 2f2cb836b5fc942e5bdba88b44f3fcf3
I20260928 14:43:37.195360 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.66:0
--local_ip_for_outbound_sockets=127.9.33.66
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:33819
--builtin_ntp_servers=127.9.33.84:32973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260928 14:43:37.197300 18630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:37.192021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:40475 (local address 127.9.33.126:33819)
0928 14:43:37.192089 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:37.192092 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:37.192364 (+ 272us) server_negotiation.cc:407] Connection header received
0928 14:43:37.192857 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:37.192858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:37.192870 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:37.192924 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:37.193599 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:37.194199 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:37.195641 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:37.195899 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:37.195981 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:37.196936 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:37.196938 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:37.196938 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:43:37.196953 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:37.196972 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:43:37.196977 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:37.197058 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:37.197181 (+ 123us) server_negotiation.cc:299] Negotiation successful
0928 14:43:37.197220 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260928 14:43:37.197542 18780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:37.192017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:33819 (local address 127.9.33.65:40475)
0928 14:43:37.192238 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:37.192274 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:37.192770 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:37.193008 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:37.193011 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:37.193020 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:37.193522 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:37.193527 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:37.194299 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:37.194300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:37.195567 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:37.195570 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:37.195981 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:37.195983 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:37.196148 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:37.196727 (+ 579us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:43:37.196728 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:37.196777 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:43:37.196813 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:37.197058 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:37.197066 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:37.197070 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:43:37.197180 (+ 110us) client_negotiation.cc:240] Negotiation successful
0928 14:43:37.197302 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":109,"threads_started":1}
I20260928 14:43:37.198071 18778 heartbeater.cc:347] Connected to a master server at 127.9.33.126:33819
I20260928 14:43:37.198185 18778 heartbeater.cc:464] Registering TS with master...
I20260928 14:43:37.198400 18778 heartbeater.cc:511] Master 127.9.33.126:33819 requested a full tablet report, sending...
I20260928 14:43:37.198937 18590 ts_manager.cc:194] Registered new tserver with Master: 72c8b456c42844b5aad1248091d6f209 (127.9.33.65:40199)
I20260928 14:43:37.199581 18590 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.33.65:40475
W20260928 14:43:37.279058 18781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:37.279330 18781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:37.279368 18781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:37.280723 18781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:37.280782 18781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:37.280829 18781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.66
I20260928 14:43:37.282218 18781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:32973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:33819
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.66
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:37.282501 18781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:37.282835 18781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:37.288089 18789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:37.288115 18787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:37.288115 18786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:37.288225 18781 server_base.cc:1034] running on GCE node
I20260928 14:43:37.288590 18781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:37.288812 18781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:37.289983 18781 hybrid_clock.cc:648] HybridClock initialized: now 1790606617289967 us; error 28 us; skew 500 ppm
I20260928 14:43:37.291272 18781 webserver.cc:460] Webserver started at http://127.9.33.66:39809/ using document root <none> and password file <none>
I20260928 14:43:37.291536 18781 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:37.291585 18781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:37.291775 18781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:37.292927 18781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/data/instance:
uuid: "2515494d4f3c411f867a66bc1e78c9cf"
format_stamp: "Formatted at 2026-09-28 14:43:37 on dist-test-slave-pv0w"
server_key: "9fe4867fd4da1cf34018c6d210fde4d2"
server_key_iv: "107d2495da664e9000d5312733864033"
server_key_version: "encryptionkey@0"
I20260928 14:43:37.293313 18781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance:
uuid: "2515494d4f3c411f867a66bc1e78c9cf"
format_stamp: "Formatted at 2026-09-28 14:43:37 on dist-test-slave-pv0w"
server_key: "9fe4867fd4da1cf34018c6d210fde4d2"
server_key_iv: "107d2495da664e9000d5312733864033"
server_key_version: "encryptionkey@0"
I20260928 14:43:37.295348 18781 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260928 14:43:37.296451 18795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:37.296741 18781 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:37.296856 18781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/wal
uuid: "2515494d4f3c411f867a66bc1e78c9cf"
format_stamp: "Formatted at 2026-09-28 14:43:37 on dist-test-slave-pv0w"
server_key: "9fe4867fd4da1cf34018c6d210fde4d2"
server_key_iv: "107d2495da664e9000d5312733864033"
server_key_version: "encryptionkey@0"
I20260928 14:43:37.296965 18781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:37.308993 18781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:37.309340 18781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:37.309499 18781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:37.309742 18781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:43:37.310101 18781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:43:37.310158 18781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:37.310204 18781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:43:37.310228 18781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:37.316349 18781 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.66:46727
I20260928 14:43:37.316401 18908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.66:46727 every 8 connection(s)
I20260928 14:43:37.316743 18781 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
I20260928 14:43:37.321220 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 18781
I20260928 14:43:37.321328 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance
I20260928 14:43:37.321589 9349 external_mini_cluster.cc:1428] Setting key b5ceac55fef036d96a32ecf83ad7cef8
I20260928 14:43:37.322350 18630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:37.317495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:56365 (local address 127.9.33.126:33819)
0928 14:43:37.317564 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:37.317566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:37.317795 (+ 229us) server_negotiation.cc:407] Connection header received
0928 14:43:37.318369 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:37.318371 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:37.318383 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:37.318439 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:37.319140 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:37.319636 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:37.320674 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:37.320868 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:37.320960 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:37.321971 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:37.321973 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:37.321973 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:43:37.321978 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:37.321996 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:43:37.322001 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:37.322062 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:37.322237 (+ 175us) server_negotiation.cc:299] Negotiation successful
0928 14:43:37.322270 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 14:43:37.322576 18911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:37.317493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:33819 (local address 127.9.33.66:56365)
0928 14:43:37.317670 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:37.317704 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:37.318251 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:37.318523 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:37.318526 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:37.318535 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:37.319036 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:37.319040 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:37.319750 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:37.319752 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:37.320568 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:37.320570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:37.320964 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:37.320965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:37.321152 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:37.321789 (+ 637us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:43:37.321790 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:37.321841 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:43:37.321868 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:37.322094 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:37.322100 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:37.322103 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:43:37.322224 (+ 121us) client_negotiation.cc:240] Negotiation successful
0928 14:43:37.322333 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":78,"threads_started":1}
I20260928 14:43:37.322665 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.67:0
--local_ip_for_outbound_sockets=127.9.33.67
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:33819
--builtin_ntp_servers=127.9.33.84:32973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260928 14:43:37.323141 18909 heartbeater.cc:347] Connected to a master server at 127.9.33.126:33819
I20260928 14:43:37.323251 18909 heartbeater.cc:464] Registering TS with master...
I20260928 14:43:37.323463 18909 heartbeater.cc:511] Master 127.9.33.126:33819 requested a full tablet report, sending...
I20260928 14:43:37.323866 18590 ts_manager.cc:194] Registered new tserver with Master: 2515494d4f3c411f867a66bc1e78c9cf (127.9.33.66:46727)
I20260928 14:43:37.324343 18590 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.33.66:56365
W20260928 14:43:37.407310 18912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:37.407567 18912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:37.407606 18912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:37.408924 18912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:37.409004 18912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:37.409046 18912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.67
I20260928 14:43:37.410530 18912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:32973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:33819
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.67
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:37.410840 18912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:37.411170 18912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:37.416718 18918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:37.416702 18920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:37.416684 18917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:37.417089 18912 server_base.cc:1034] running on GCE node
I20260928 14:43:37.417408 18912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:37.417689 18912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:37.418917 18912 hybrid_clock.cc:648] HybridClock initialized: now 1790606617418885 us; error 43 us; skew 500 ppm
I20260928 14:43:37.420480 18912 webserver.cc:460] Webserver started at http://127.9.33.67:42441/ using document root <none> and password file <none>
I20260928 14:43:37.420755 18912 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:37.420843 18912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:37.421018 18912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:37.422294 18912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/data/instance:
uuid: "c7b999c22e294be7ab9ad839051506f5"
format_stamp: "Formatted at 2026-09-28 14:43:37 on dist-test-slave-pv0w"
server_key: "37f1eb935a02b8c7a85d0317f1db4c9a"
server_key_iv: "9f5eaca5eaf410692431edf7572f0be4"
server_key_version: "encryptionkey@0"
I20260928 14:43:37.422667 18912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance:
uuid: "c7b999c22e294be7ab9ad839051506f5"
format_stamp: "Formatted at 2026-09-28 14:43:37 on dist-test-slave-pv0w"
server_key: "37f1eb935a02b8c7a85d0317f1db4c9a"
server_key_iv: "9f5eaca5eaf410692431edf7572f0be4"
server_key_version: "encryptionkey@0"
I20260928 14:43:37.424280 18912 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260928 14:43:37.425345 18926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:37.425594 18912 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:37.425711 18912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/wal
uuid: "c7b999c22e294be7ab9ad839051506f5"
format_stamp: "Formatted at 2026-09-28 14:43:37 on dist-test-slave-pv0w"
server_key: "37f1eb935a02b8c7a85d0317f1db4c9a"
server_key_iv: "9f5eaca5eaf410692431edf7572f0be4"
server_key_version: "encryptionkey@0"
I20260928 14:43:37.425791 18912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:37.440986 18912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:37.441347 18912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:37.441510 18912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:37.441752 18912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:43:37.442121 18912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:43:37.442176 18912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:37.442217 18912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:43:37.442255 18912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:37.448179 18912 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.67:42709
I20260928 14:43:37.448222 19039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.67:42709 every 8 connection(s)
I20260928 14:43:37.448567 18912 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
I20260928 14:43:37.454273 18630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:37.449661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:34319 (local address 127.9.33.126:33819)
0928 14:43:37.449703 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:37.449705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:37.449807 (+ 102us) server_negotiation.cc:407] Connection header received
0928 14:43:37.450540 (+ 733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:37.450542 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:37.450555 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:37.450620 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:37.451308 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:37.451916 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:37.452953 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:37.453165 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:37.453229 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:37.454043 (+ 814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:37.454044 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:37.454045 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:43:37.454049 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:37.454065 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:43:37.454069 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:37.454117 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:37.454185 (+ 68us) server_negotiation.cc:299] Negotiation successful
0928 14:43:37.454211 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 14:43:37.454490 19042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:37.449320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:33819 (local address 127.9.33.67:34319)
0928 14:43:37.449601 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:37.449680 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:37.450425 (+ 745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:37.450678 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:37.450681 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:37.450690 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:37.451206 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:37.451210 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:37.452033 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:37.452034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:37.452840 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:37.452843 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:37.453249 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:37.453250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:37.453390 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:37.453906 (+ 516us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:43:37.453906 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:37.453936 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:43:37.453958 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:37.454119 (+ 161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:37.454122 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:37.454124 (+ 2us) client_negotiation.cc:769] Sending connection context
0928 14:43:37.454173 (+ 49us) client_negotiation.cc:240] Negotiation successful
0928 14:43:37.454254 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":107,"threads_started":1}
I20260928 14:43:37.455019 19040 heartbeater.cc:347] Connected to a master server at 127.9.33.126:33819
I20260928 14:43:37.455112 19040 heartbeater.cc:464] Registering TS with master...
I20260928 14:43:37.455263 19040 heartbeater.cc:511] Master 127.9.33.126:33819 requested a full tablet report, sending...
I20260928 14:43:37.455650 18590 ts_manager.cc:194] Registered new tserver with Master: c7b999c22e294be7ab9ad839051506f5 (127.9.33.67:42709)
I20260928 14:43:37.456036 18590 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.33.67:34319
I20260928 14:43:37.458087 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 18912
I20260928 14:43:37.458170 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance
I20260928 14:43:37.458344 9349 external_mini_cluster.cc:1428] Setting key 1ddbc1b9702892ed8277293ddbf166b0
I20260928 14:43:37.459230 9349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260928 14:43:37.461262 18630 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:37.460656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55638 (local address 127.9.33.126:33819)
0928 14:43:37.460727 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:37.460730 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:37.460919 (+ 189us) server_negotiation.cc:407] Connection header received
0928 14:43:37.460951 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:37.460952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:37.460964 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:37.461020 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:37.461195 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55638: BlockingRecv error: recv got EOF from 127.0.0.1:55638 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260928 14:43:37.461293 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 18650
I20260928 14:43:37.466166 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 18781
I20260928 14:43:37.470322 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 18912
I20260928 14:43:37.474436 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 18559
2026-09-28T14:43:37Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (609 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-28T14:43:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:43:37Z Disabled control of system clock
I20260928 14:43:37.501557 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:36367
--webserver_interface=127.9.33.126
--webserver_port=0
--builtin_ntp_servers=127.9.33.84:43425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:36367
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260928 14:43:37.582233 19051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:37.582472 19051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:37.582494 19051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:37.583612 19051 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:43:37.583652 19051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:37.583665 19051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:37.583678 19051 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:43:37.583690 19051 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:43:37.585124 19051 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:43425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:36367
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:36367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:37.585372 19051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:37.585675 19051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 14:43:37.590823 19051 server_base.cc:1034] running on GCE node
W20260928 14:43:37.590768 19056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:37.590746 19059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:37.590801 19057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:37.591246 19051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:37.591548 19051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:37.592712 19051 hybrid_clock.cc:648] HybridClock initialized: now 1790606617592716 us; error 45 us; skew 500 ppm
I20260928 14:43:37.593937 19051 webserver.cc:460] Webserver started at http://127.9.33.126:46721/ using document root <none> and password file <none>
I20260928 14:43:37.594195 19051 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:37.594264 19051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:37.594416 19051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:37.595582 19051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "83a44c23b38d4673b00440ab61cf7df8"
format_stamp: "Formatted at 2026-09-28 14:43:37 on dist-test-slave-pv0w"
server_key: "dceb23a113753f663e633cbe3e1d8c60"
server_key_iv: "22632816623b192e71d8d5a664015554"
server_key_version: "encryptionkey@0"
I20260928 14:43:37.596032 19051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "83a44c23b38d4673b00440ab61cf7df8"
format_stamp: "Formatted at 2026-09-28 14:43:37 on dist-test-slave-pv0w"
server_key: "dceb23a113753f663e633cbe3e1d8c60"
server_key_iv: "22632816623b192e71d8d5a664015554"
server_key_version: "encryptionkey@0"
I20260928 14:43:37.597431 19051 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:37.598305 19065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:37.598518 19051 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:37.598623 19051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "83a44c23b38d4673b00440ab61cf7df8"
format_stamp: "Formatted at 2026-09-28 14:43:37 on dist-test-slave-pv0w"
server_key: "dceb23a113753f663e633cbe3e1d8c60"
server_key_iv: "22632816623b192e71d8d5a664015554"
server_key_version: "encryptionkey@0"
I20260928 14:43:37.598707 19051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:37.602135 19051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:37.602413 19051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:37.602559 19051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:37.606598 19051 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:36367
I20260928 14:43:37.606630 19117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:36367 every 8 connection(s)
I20260928 14:43:37.606984 19051 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:43:37.607622 19118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:37.610423 19118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83a44c23b38d4673b00440ab61cf7df8: Bootstrap starting.
I20260928 14:43:37.611037 19118 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83a44c23b38d4673b00440ab61cf7df8: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:37.611341 19118 log.cc:824] T 00000000000000000000000000000000 P 83a44c23b38d4673b00440ab61cf7df8: Log is configured to *not* fsync() on all Append() calls
I20260928 14:43:37.612071 19118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83a44c23b38d4673b00440ab61cf7df8: No bootstrap required, opened a new log
I20260928 14:43:37.613651 19118 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83a44c23b38d4673b00440ab61cf7df8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83a44c23b38d4673b00440ab61cf7df8" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 36367 } }
I20260928 14:43:37.613838 19118 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83a44c23b38d4673b00440ab61cf7df8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:37.613912 19118 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83a44c23b38d4673b00440ab61cf7df8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83a44c23b38d4673b00440ab61cf7df8, State: Initialized, Role: FOLLOWER
I20260928 14:43:37.614063 19118 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83a44c23b38d4673b00440ab61cf7df8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83a44c23b38d4673b00440ab61cf7df8" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 36367 } }
I20260928 14:43:37.614148 19118 raft_consensus.cc:388] T 00000000000000000000000000000000 P 83a44c23b38d4673b00440ab61cf7df8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:43:37.614171 19118 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83a44c23b38d4673b00440ab61cf7df8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:43:37.614198 19118 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83a44c23b38d4673b00440ab61cf7df8 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:37.614699 19118 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83a44c23b38d4673b00440ab61cf7df8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83a44c23b38d4673b00440ab61cf7df8" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 36367 } }
I20260928 14:43:37.614784 19118 leader_election.cc:302] T 00000000000000000000000000000000 P 83a44c23b38d4673b00440ab61cf7df8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 83a44c23b38d4673b00440ab61cf7df8; no voters:
I20260928 14:43:37.614920 19118 leader_election.cc:288] T 00000000000000000000000000000000 P 83a44c23b38d4673b00440ab61cf7df8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:43:37.615002 19121 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83a44c23b38d4673b00440ab61cf7df8 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:43:37.615159 19121 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83a44c23b38d4673b00440ab61cf7df8 [term 1 LEADER]: Becoming Leader. State: Replica: 83a44c23b38d4673b00440ab61cf7df8, State: Running, Role: LEADER
I20260928 14:43:37.615198 19118 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83a44c23b38d4673b00440ab61cf7df8 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:43:37.615249 19121 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83a44c23b38d4673b00440ab61cf7df8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83a44c23b38d4673b00440ab61cf7df8" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 36367 } }
I20260928 14:43:37.615689 19122 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83a44c23b38d4673b00440ab61cf7df8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "83a44c23b38d4673b00440ab61cf7df8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83a44c23b38d4673b00440ab61cf7df8" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 36367 } } }
I20260928 14:43:37.615896 19122 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83a44c23b38d4673b00440ab61cf7df8 [sys.catalog]: This master's current role is: LEADER
I20260928 14:43:37.615917 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 19051
I20260928 14:43:37.616005 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:43:37.616221 9349 external_mini_cluster.cc:1428] Setting key f6c1098b395f154c144916941437a64a
I20260928 14:43:37.616384 19130 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:43:37.616997 19123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83a44c23b38d4673b00440ab61cf7df8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83a44c23b38d4673b00440ab61cf7df8. Latest consensus state: current_term: 1 leader_uuid: "83a44c23b38d4673b00440ab61cf7df8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "83a44c23b38d4673b00440ab61cf7df8" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 36367 } } }
I20260928 14:43:37.617074 19123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83a44c23b38d4673b00440ab61cf7df8 [sys.catalog]: This master's current role is: LEADER
I20260928 14:43:37.617192 19130 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:43:37.618698 19130 catalog_manager.cc:1347] Generated new cluster ID: 2969663711e64e7395a3d3a1fcf08b46
I20260928 14:43:37.618731 19139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:37.616692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34816 (local address 127.9.33.126:36367)
0928 14:43:37.617003 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:37.617012 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:37.617050 (+ 38us) server_negotiation.cc:407] Connection header received
0928 14:43:37.617170 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:37.617177 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:37.617222 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:37.617361 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:37.618236 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:37.618240 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:37.618241 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:43:37.618245 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:37.618289 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:43:37.618304 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:37.618405 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:37.618481 (+ 76us) server_negotiation.cc:299] Negotiation successful
0928 14:43:37.618538 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":72,"threads_started":1}
I20260928 14:43:37.618749 19130 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:43:37.624517 19130 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:43:37.624701 19130 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:43:37.635659 19130 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83a44c23b38d4673b00440ab61cf7df8: Generated new TSK 0
I20260928 14:43:37.635960 19130 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:43:37.638705 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.65:0
--local_ip_for_outbound_sockets=127.9.33.65
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:36367
--builtin_ntp_servers=127.9.33.84:43425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260928 14:43:37.723130 19142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:37.723366 19142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:37.723416 19142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:37.724686 19142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:37.724730 19142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:37.724757 19142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.65
I20260928 14:43:37.726150 19142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:43425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:36367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.65
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:37.726397 19142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:37.726732 19142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:37.732838 19148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:37.732842 19147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:37.732822 19150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:37.733134 19142 server_base.cc:1034] running on GCE node
I20260928 14:43:37.733433 19142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:37.733713 19142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:37.734884 19142 hybrid_clock.cc:648] HybridClock initialized: now 1790606617734861 us; error 40 us; skew 500 ppm
I20260928 14:43:37.736497 19142 webserver.cc:460] Webserver started at http://127.9.33.65:33485/ using document root <none> and password file <none>
I20260928 14:43:37.736757 19142 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:37.736824 19142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:37.736982 19142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:37.738421 19142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/data/instance:
uuid: "972b814945eb40c0a4e09c08ff246827"
format_stamp: "Formatted at 2026-09-28 14:43:37 on dist-test-slave-pv0w"
server_key: "73b5af055aee7b763272b295d6233764"
server_key_iv: "d3d80f2e3c448f2aa92feac0dabcacc0"
server_key_version: "encryptionkey@0"
I20260928 14:43:37.738831 19142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance:
uuid: "972b814945eb40c0a4e09c08ff246827"
format_stamp: "Formatted at 2026-09-28 14:43:37 on dist-test-slave-pv0w"
server_key: "73b5af055aee7b763272b295d6233764"
server_key_iv: "d3d80f2e3c448f2aa92feac0dabcacc0"
server_key_version: "encryptionkey@0"
I20260928 14:43:37.740676 19142 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260928 14:43:37.741572 19156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:37.741778 19142 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:37.741870 19142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/wal
uuid: "972b814945eb40c0a4e09c08ff246827"
format_stamp: "Formatted at 2026-09-28 14:43:37 on dist-test-slave-pv0w"
server_key: "73b5af055aee7b763272b295d6233764"
server_key_iv: "d3d80f2e3c448f2aa92feac0dabcacc0"
server_key_version: "encryptionkey@0"
I20260928 14:43:37.741935 19142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:37.746389 19142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:37.746657 19142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:37.746776 19142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:37.746974 19142 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:43:37.747342 19142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:43:37.747385 19142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:37.747416 19142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:43:37.747442 19142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:37.754122 19142 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.65:33283
I20260928 14:43:37.754168 19269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.65:33283 every 8 connection(s)
I20260928 14:43:37.754537 19142 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
I20260928 14:43:37.757644 19139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:37.755272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:37711 (local address 127.9.33.126:36367)
0928 14:43:37.755315 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:37.755317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:37.755556 (+ 239us) server_negotiation.cc:407] Connection header received
0928 14:43:37.756127 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:37.756128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:37.756135 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:37.756207 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:37.757420 (+ 1213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:37.757422 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:37.757422 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:43:37.757426 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:37.757442 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:43:37.757447 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:37.757502 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:37.757577 (+ 75us) server_negotiation.cc:299] Negotiation successful
0928 14:43:37.757590 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260928 14:43:37.757844 19272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:37.755240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:36367 (local address 127.9.33.65:37711)
0928 14:43:37.755427 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:37.755479 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:37.756025 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:37.756283 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:37.756288 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:37.756296 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:37.757276 (+ 980us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:43:37.757277 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:37.757308 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:43:37.757336 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:37.757519 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:37.757523 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:37.757526 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:43:37.757579 (+ 53us) client_negotiation.cc:240] Negotiation successful
0928 14:43:37.757630 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":93,"threads_started":1}
I20260928 14:43:37.758369 19270 heartbeater.cc:347] Connected to a master server at 127.9.33.126:36367
I20260928 14:43:37.758471 19270 heartbeater.cc:464] Registering TS with master...
I20260928 14:43:37.758589 19270 heartbeater.cc:511] Master 127.9.33.126:36367 requested a full tablet report, sending...
I20260928 14:43:37.759086 19082 ts_manager.cc:194] Registered new tserver with Master: 972b814945eb40c0a4e09c08ff246827 (127.9.33.65:33283)
I20260928 14:43:37.764108 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 19142
I20260928 14:43:37.764191 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance
I20260928 14:43:37.764348 9349 external_mini_cluster.cc:1428] Setting key 599f852f70c4515c185898bffc091d4e
I20260928 14:43:37.765425 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.66:0
--local_ip_for_outbound_sockets=127.9.33.66
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:36367
--builtin_ntp_servers=127.9.33.84:43425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260928 14:43:37.849052 19273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:37.849267 19273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:37.849289 19273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:37.850505 19273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:37.850546 19273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:37.850572 19273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.66
I20260928 14:43:37.851960 19273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:43425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:36367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.66
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:37.852151 19273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:37.852396 19273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:37.857739 19279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:37.857745 19281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:37.857836 19273 server_base.cc:1034] running on GCE node
W20260928 14:43:37.857745 19278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:37.858201 19273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:37.858412 19273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:37.859584 19273 hybrid_clock.cc:648] HybridClock initialized: now 1790606617859579 us; error 42 us; skew 500 ppm
I20260928 14:43:37.860828 19273 webserver.cc:460] Webserver started at http://127.9.33.66:42885/ using document root <none> and password file <none>
I20260928 14:43:37.861099 19273 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:37.861207 19273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:37.861372 19273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:37.862488 19273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/data/instance:
uuid: "b2c13c81d8774bdf87ab00694b7660ca"
format_stamp: "Formatted at 2026-09-28 14:43:37 on dist-test-slave-pv0w"
server_key: "cb7ce5bf3381751892d79583b6cc4fb8"
server_key_iv: "2581e069250fd03cdc6be1e0d9e1b90c"
server_key_version: "encryptionkey@0"
I20260928 14:43:37.862885 19273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance:
uuid: "b2c13c81d8774bdf87ab00694b7660ca"
format_stamp: "Formatted at 2026-09-28 14:43:37 on dist-test-slave-pv0w"
server_key: "cb7ce5bf3381751892d79583b6cc4fb8"
server_key_iv: "2581e069250fd03cdc6be1e0d9e1b90c"
server_key_version: "encryptionkey@0"
I20260928 14:43:37.864470 19273 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260928 14:43:37.865243 19287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:37.865430 19273 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260928 14:43:37.865545 19273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/wal
uuid: "b2c13c81d8774bdf87ab00694b7660ca"
format_stamp: "Formatted at 2026-09-28 14:43:37 on dist-test-slave-pv0w"
server_key: "cb7ce5bf3381751892d79583b6cc4fb8"
server_key_iv: "2581e069250fd03cdc6be1e0d9e1b90c"
server_key_version: "encryptionkey@0"
I20260928 14:43:37.865621 19273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:37.869117 19273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:37.869401 19273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:37.869552 19273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:37.869761 19273 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:43:37.870071 19273 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:43:37.870126 19273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:37.870167 19273 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:43:37.870194 19273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:37.876289 19273 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.66:35381
I20260928 14:43:37.876441 19400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.66:35381 every 8 connection(s)
I20260928 14:43:37.876735 19273 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
I20260928 14:43:37.880059 19139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:37.877570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:55195 (local address 127.9.33.126:36367)
0928 14:43:37.877620 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:37.877622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:37.877891 (+ 269us) server_negotiation.cc:407] Connection header received
0928 14:43:37.878381 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:37.878382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:37.878386 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:37.878452 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:37.879819 (+ 1367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:37.879821 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:37.879822 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:43:37.879827 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:37.879846 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:43:37.879850 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:37.879907 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:37.879991 (+ 84us) server_negotiation.cc:299] Negotiation successful
0928 14:43:37.880004 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 14:43:37.880275 19403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:37.877538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:36367 (local address 127.9.33.66:55195)
0928 14:43:37.877748 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:37.877795 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:37.878292 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:37.878509 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:37.878513 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:37.878522 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:37.879614 (+ 1092us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:43:37.879615 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:37.879648 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:43:37.879681 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:37.879915 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:37.879919 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:37.879922 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:43:37.879984 (+ 62us) client_negotiation.cc:240] Negotiation successful
0928 14:43:37.880043 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":108,"threads_started":1}
I20260928 14:43:37.880466 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 19273
I20260928 14:43:37.880585 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance
I20260928 14:43:37.880815 9349 external_mini_cluster.cc:1428] Setting key e156cf9519ab5f32b8fdbfa99ce66592
I20260928 14:43:37.880894 19401 heartbeater.cc:347] Connected to a master server at 127.9.33.126:36367
I20260928 14:43:37.881014 19401 heartbeater.cc:464] Registering TS with master...
I20260928 14:43:37.881204 19401 heartbeater.cc:511] Master 127.9.33.126:36367 requested a full tablet report, sending...
I20260928 14:43:37.881647 19082 ts_manager.cc:194] Registered new tserver with Master: b2c13c81d8774bdf87ab00694b7660ca (127.9.33.66:35381)
I20260928 14:43:37.881901 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.67:0
--local_ip_for_outbound_sockets=127.9.33.67
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:36367
--builtin_ntp_servers=127.9.33.84:43425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260928 14:43:37.969460 19404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:37.969669 19404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:37.969692 19404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:37.970818 19404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:37.970857 19404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:37.970882 19404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.67
I20260928 14:43:37.972218 19404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:43425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:36367
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.67
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:37.972419 19404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:37.972661 19404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:37.977708 19412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:37.977813 19410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:37.977725 19409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:37.977994 19404 server_base.cc:1034] running on GCE node
I20260928 14:43:37.978242 19404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:37.978540 19404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:37.979694 19404 hybrid_clock.cc:648] HybridClock initialized: now 1790606617979675 us; error 39 us; skew 500 ppm
I20260928 14:43:37.981231 19404 webserver.cc:460] Webserver started at http://127.9.33.67:39619/ using document root <none> and password file <none>
I20260928 14:43:37.981539 19404 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:37.981628 19404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:37.981796 19404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:37.982985 19404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/data/instance:
uuid: "182d2a95e9ee41a5a68c096d8ce0f18b"
format_stamp: "Formatted at 2026-09-28 14:43:37 on dist-test-slave-pv0w"
server_key: "ba269e4621f89c8859cc1d13575b7f75"
server_key_iv: "f9ce977186c083704a59831615166cda"
server_key_version: "encryptionkey@0"
I20260928 14:43:37.983382 19404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance:
uuid: "182d2a95e9ee41a5a68c096d8ce0f18b"
format_stamp: "Formatted at 2026-09-28 14:43:37 on dist-test-slave-pv0w"
server_key: "ba269e4621f89c8859cc1d13575b7f75"
server_key_iv: "f9ce977186c083704a59831615166cda"
server_key_version: "encryptionkey@0"
I20260928 14:43:37.984892 19404 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:43:37.985726 19418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:37.985946 19404 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:37.986053 19404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/wal
uuid: "182d2a95e9ee41a5a68c096d8ce0f18b"
format_stamp: "Formatted at 2026-09-28 14:43:37 on dist-test-slave-pv0w"
server_key: "ba269e4621f89c8859cc1d13575b7f75"
server_key_iv: "f9ce977186c083704a59831615166cda"
server_key_version: "encryptionkey@0"
I20260928 14:43:37.986150 19404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:37.990048 19404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:37.990335 19404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:37.990495 19404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:37.990710 19404 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:43:37.991047 19404 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:43:37.991112 19404 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:37.991155 19404 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:43:37.991186 19404 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:37.997673 19404 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.67:35945
I20260928 14:43:37.997715 19531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.67:35945 every 8 connection(s)
I20260928 14:43:37.998075 19404 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
I20260928 14:43:38.001670 19139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:37.998857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:52395 (local address 127.9.33.126:36367)
0928 14:43:37.998929 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:37.998932 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:37.999261 (+ 329us) server_negotiation.cc:407] Connection header received
0928 14:43:37.999895 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:37.999897 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:37.999905 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:37.999974 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:38.001376 (+ 1402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:38.001378 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:38.001378 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:43:38.001383 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:38.001401 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:43:38.001406 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:38.001467 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:38.001571 (+ 104us) server_negotiation.cc:299] Negotiation successful
0928 14:43:38.001590 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 14:43:38.001904 19534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:37.998842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:36367 (local address 127.9.33.67:52395)
0928 14:43:37.999074 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:37.999148 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:37.999689 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:38.000063 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:38.000067 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:38.000075 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:38.001192 (+ 1117us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:43:38.001192 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:38.001241 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:43:38.001274 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:38.001469 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:38.001475 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:38.001478 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:43:38.001588 (+ 110us) client_negotiation.cc:240] Negotiation successful
0928 14:43:38.001655 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":118,"threads_started":1}
I20260928 14:43:38.002480 19532 heartbeater.cc:347] Connected to a master server at 127.9.33.126:36367
I20260928 14:43:38.002591 19532 heartbeater.cc:464] Registering TS with master...
I20260928 14:43:38.002758 19532 heartbeater.cc:511] Master 127.9.33.126:36367 requested a full tablet report, sending...
I20260928 14:43:38.003172 19082 ts_manager.cc:194] Registered new tserver with Master: 182d2a95e9ee41a5a68c096d8ce0f18b (127.9.33.67:35945)
I20260928 14:43:38.006882 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 19404
I20260928 14:43:38.006973 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance
I20260928 14:43:38.007151 9349 external_mini_cluster.cc:1428] Setting key 900cb46c0bd2b6a273e637397d71555f
I20260928 14:43:38.008514 9349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260928 14:43:38.012327 19139 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:43:38.010369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34832 (local address 127.9.33.126:36367)
0928 14:43:38.010454 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:38.010456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:38.011899 (+ 1443us) server_negotiation.cc:407] Connection header received
0928 14:43:38.012017 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:38.012017 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:38.012038 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:38.012090 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:38.012250 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34832: BlockingRecv error: recv got EOF from 127.0.0.1:34832 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 14:43:38.012590 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 19142
I20260928 14:43:38.019476 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 19273
I20260928 14:43:38.024426 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 19404
I20260928 14:43:38.029422 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 19051
2026-09-28T14:43:38Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (555 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:43:38 dist-test-slave-pv0w krb5kdc[19542](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:43:38 dist-test-slave-pv0w krb5kdc[19542](info): set up 2 sockets
Sep 28 14:43:38 dist-test-slave-pv0w krb5kdc[19542](info): commencing operation
krb5kdc: starting...
W20260928 14:43:39.097342 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.048s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:43:39 dist-test-slave-pv0w krb5kdc[19542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:43:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:43:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.33.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
I20260928 14:43:39.224124 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:33187
--webserver_interface=127.9.33.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:44105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:33187
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:43:39.306388 19555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:39.306653 19555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:39.306692 19555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:39.307968 19555 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:43:39.308032 19555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:39.308055 19555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:39.308085 19555 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:43:39.308116 19555 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:43:39.309613 19555 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:44105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:33187
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:33187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:39.309893 19555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:39.310134 19555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:39.315294 19560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:39.315366 19563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:39.315304 19561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:39.315438 19555 server_base.cc:1034] running on GCE node
I20260928 14:43:39.315834 19555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:39.316107 19555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:39.317279 19555 hybrid_clock.cc:648] HybridClock initialized: now 1790606619317274 us; error 40 us; skew 500 ppm
Sep 28 14:43:39 dist-test-slave-pv0w krb5kdc[19542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606619, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:39.319938 19555 init.cc:377] Logged in from keytab as kudu/127.9.33.126@KRBTEST.COM (short username kudu)
I20260928 14:43:39.320420 19555 webserver.cc:460] Webserver started at http://127.9.33.126:33051/ using document root <none> and password file <none>
I20260928 14:43:39.320618 19555 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:39.320658 19555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:39.320751 19555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:39.321843 19555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "130bf81aec574e71a6751916b68d53bb"
format_stamp: "Formatted at 2026-09-28 14:43:39 on dist-test-slave-pv0w"
server_key: "14c8871563f104578e627a99874577f1"
server_key_iv: "0b4e97dc50e567d94cb01ebf479b7c71"
server_key_version: "encryptionkey@0"
I20260928 14:43:39.322157 19555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "130bf81aec574e71a6751916b68d53bb"
format_stamp: "Formatted at 2026-09-28 14:43:39 on dist-test-slave-pv0w"
server_key: "14c8871563f104578e627a99874577f1"
server_key_iv: "0b4e97dc50e567d94cb01ebf479b7c71"
server_key_version: "encryptionkey@0"
I20260928 14:43:39.323472 19555 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:43:39.324240 19570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:39.324401 19555 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:39.324501 19555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "130bf81aec574e71a6751916b68d53bb"
format_stamp: "Formatted at 2026-09-28 14:43:39 on dist-test-slave-pv0w"
server_key: "14c8871563f104578e627a99874577f1"
server_key_iv: "0b4e97dc50e567d94cb01ebf479b7c71"
server_key_version: "encryptionkey@0"
I20260928 14:43:39.324559 19555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:39.339989 19555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:39.342406 19555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:39.342528 19555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:39.346556 19555 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:33187
I20260928 14:43:39.346626 19622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:33187 every 8 connection(s)
I20260928 14:43:39.347013 19555 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:43:39.348162 19623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:39.349577 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 19555
I20260928 14:43:39.349663 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:43:39.349884 9349 external_mini_cluster.cc:1428] Setting key 3ee2ad3f49db2e7da44850b3ad6f5ddb
I20260928 14:43:39.351094 19623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 130bf81aec574e71a6751916b68d53bb: Bootstrap starting.
I20260928 14:43:39.351900 19623 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 130bf81aec574e71a6751916b68d53bb: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:39.352211 19623 log.cc:824] T 00000000000000000000000000000000 P 130bf81aec574e71a6751916b68d53bb: Log is configured to *not* fsync() on all Append() calls
I20260928 14:43:39.353012 19623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 130bf81aec574e71a6751916b68d53bb: No bootstrap required, opened a new log
I20260928 14:43:39.355297 19623 raft_consensus.cc:348] T 00000000000000000000000000000000 P 130bf81aec574e71a6751916b68d53bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "130bf81aec574e71a6751916b68d53bb" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 33187 } }
I20260928 14:43:39.355489 19623 raft_consensus.cc:374] T 00000000000000000000000000000000 P 130bf81aec574e71a6751916b68d53bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:39.355532 19623 raft_consensus.cc:729] T 00000000000000000000000000000000 P 130bf81aec574e71a6751916b68d53bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130bf81aec574e71a6751916b68d53bb, State: Initialized, Role: FOLLOWER
I20260928 14:43:39.355697 19623 consensus_queue.cc:260] T 00000000000000000000000000000000 P 130bf81aec574e71a6751916b68d53bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "130bf81aec574e71a6751916b68d53bb" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 33187 } }
I20260928 14:43:39.355825 19623 raft_consensus.cc:388] T 00000000000000000000000000000000 P 130bf81aec574e71a6751916b68d53bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:43:39.355851 19623 raft_consensus.cc:482] T 00000000000000000000000000000000 P 130bf81aec574e71a6751916b68d53bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:43:39.355880 19623 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 130bf81aec574e71a6751916b68d53bb [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:39.356515 19623 raft_consensus.cc:504] T 00000000000000000000000000000000 P 130bf81aec574e71a6751916b68d53bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "130bf81aec574e71a6751916b68d53bb" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 33187 } }
I20260928 14:43:39.356603 19623 leader_election.cc:302] T 00000000000000000000000000000000 P 130bf81aec574e71a6751916b68d53bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 130bf81aec574e71a6751916b68d53bb; no voters:
I20260928 14:43:39.356822 19623 leader_election.cc:288] T 00000000000000000000000000000000 P 130bf81aec574e71a6751916b68d53bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:43:39.356850 19628 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 130bf81aec574e71a6751916b68d53bb [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:43:39.356966 19628 raft_consensus.cc:686] T 00000000000000000000000000000000 P 130bf81aec574e71a6751916b68d53bb [term 1 LEADER]: Becoming Leader. State: Replica: 130bf81aec574e71a6751916b68d53bb, State: Running, Role: LEADER
I20260928 14:43:39.357107 19623 sys_catalog.cc:564] T 00000000000000000000000000000000 P 130bf81aec574e71a6751916b68d53bb [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:43:39.357117 19628 consensus_queue.cc:237] T 00000000000000000000000000000000 P 130bf81aec574e71a6751916b68d53bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "130bf81aec574e71a6751916b68d53bb" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 33187 } }
Sep 28 14:43:39 dist-test-slave-pv0w krb5kdc[19542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:43:39.357601 19630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 130bf81aec574e71a6751916b68d53bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 130bf81aec574e71a6751916b68d53bb. Latest consensus state: current_term: 1 leader_uuid: "130bf81aec574e71a6751916b68d53bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "130bf81aec574e71a6751916b68d53bb" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 33187 } } }
I20260928 14:43:39.357692 19630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 130bf81aec574e71a6751916b68d53bb [sys.catalog]: This master's current role is: LEADER
I20260928 14:43:39.357643 19629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 130bf81aec574e71a6751916b68d53bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "130bf81aec574e71a6751916b68d53bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "130bf81aec574e71a6751916b68d53bb" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 33187 } } }
I20260928 14:43:39.357733 19629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 130bf81aec574e71a6751916b68d53bb [sys.catalog]: This master's current role is: LEADER
I20260928 14:43:39.358603 19636 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:43:39.359191 19636 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:43:39.360996 19636 catalog_manager.cc:1347] Generated new cluster ID: d95ee2d70fc049ae9fc9f0ccb04e6020
I20260928 14:43:39.361050 19636 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:43:39.362238 19626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:39.350485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55668 (local address 127.9.33.126:33187)
0928 14:43:39.350711 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:39.350722 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:39.350755 (+ 33us) server_negotiation.cc:407] Connection header received
0928 14:43:39.351280 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:39.351290 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:39.351352 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:39.351535 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:39.352327 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:39.353702 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:39.354528 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:39.355262 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:39.355381 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:39.358145 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:39.358150 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:39.358150 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:39.358164 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:39.360377 (+ 2213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:39.360814 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:39.360817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:39.360818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:39.360991 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:39.361291 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:39.361292 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:39.361292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:39.361615 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:39.361765 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:39.361887 (+ 122us) server_negotiation.cc:299] Negotiation successful
0928 14:43:39.362039 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":105,"threads_started":1}
I20260928 14:43:39.369948 19636 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:43:39.370699 19636 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:43:39.375142 19636 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 130bf81aec574e71a6751916b68d53bb: Generated new TSK 0
I20260928 14:43:39.375344 19636 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.33.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab.
Entry for principal kudu/127.9.33.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab.
I20260928 14:43:39.410799 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.65:0
--local_ip_for_outbound_sockets=127.9.33.65
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:33187
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:44105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:43:39.493948 19649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:39.494158 19649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:39.494182 19649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:39.495378 19649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:39.495419 19649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:39.495487 19649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.65
I20260928 14:43:39.497023 19649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:44105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:33187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.65
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:39.497220 19649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:39.497397 19649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:39.502638 19655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:39.502748 19649 server_base.cc:1034] running on GCE node
W20260928 14:43:39.502657 19654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:39.502657 19657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:39.503087 19649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:39.503293 19649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:39.504439 19649 hybrid_clock.cc:648] HybridClock initialized: now 1790606619504422 us; error 34 us; skew 500 ppm
Sep 28 14:43:39 dist-test-slave-pv0w krb5kdc[19542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606619, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:39.506927 19649 init.cc:377] Logged in from keytab as kudu/127.9.33.65@KRBTEST.COM (short username kudu)
I20260928 14:43:39.507362 19649 webserver.cc:460] Webserver started at http://127.9.33.65:41679/ using document root <none> and password file <none>
I20260928 14:43:39.507598 19649 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:39.507668 19649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:39.507849 19649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:39.508968 19649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/data/instance:
uuid: "7ba297d9e1bf42eead0ad93c2d5fb159"
format_stamp: "Formatted at 2026-09-28 14:43:39 on dist-test-slave-pv0w"
server_key: "545dfa210baf299b080c37e4f91610e2"
server_key_iv: "0488b258d23b56fece1fa2e0d6323a00"
server_key_version: "encryptionkey@0"
I20260928 14:43:39.509335 19649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance:
uuid: "7ba297d9e1bf42eead0ad93c2d5fb159"
format_stamp: "Formatted at 2026-09-28 14:43:39 on dist-test-slave-pv0w"
server_key: "545dfa210baf299b080c37e4f91610e2"
server_key_iv: "0488b258d23b56fece1fa2e0d6323a00"
server_key_version: "encryptionkey@0"
I20260928 14:43:39.510679 19649 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:43:39.511479 19664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:39.511654 19649 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:43:39.511782 19649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/wal
uuid: "7ba297d9e1bf42eead0ad93c2d5fb159"
format_stamp: "Formatted at 2026-09-28 14:43:39 on dist-test-slave-pv0w"
server_key: "545dfa210baf299b080c37e4f91610e2"
server_key_iv: "0488b258d23b56fece1fa2e0d6323a00"
server_key_version: "encryptionkey@0"
I20260928 14:43:39.511853 19649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:39.537102 19649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:39.539009 19649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:39.539124 19649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:39.539309 19649 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:43:39.539646 19649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:43:39.539682 19649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:39.539705 19649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:43:39.539776 19649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:39.545972 19649 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.65:41983
I20260928 14:43:39.546017 19777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.65:41983 every 8 connection(s)
I20260928 14:43:39.546371 19649 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
Sep 28 14:43:39 dist-test-slave-pv0w krb5kdc[19542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606619, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:43:39.555908 19626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:39.547210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:35127 (local address 127.9.33.126:33187)
0928 14:43:39.547288 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:39.547291 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:39.547531 (+ 240us) server_negotiation.cc:407] Connection header received
0928 14:43:39.548062 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:39.548063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:39.548073 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:39.548127 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:39.549115 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:39.549762 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:39.550788 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:39.551006 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:39.551087 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:39.553264 (+ 2177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:39.553265 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:39.553265 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:39.553272 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:39.554694 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:39.555054 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:39.555055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:39.555055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:39.555107 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:39.555319 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:39.555320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:39.555320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:39.555420 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:39.555485 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:39.555766 (+ 281us) server_negotiation.cc:299] Negotiation successful
0928 14:43:39.555838 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260928 14:43:39.556267 19780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:39.547174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:33187 (local address 127.9.33.65:35127)
0928 14:43:39.547350 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:39.547390 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:39.547966 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:39.548183 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:39.548187 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:39.548792 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:39.549003 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:39.549008 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:39.549843 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:39.549844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:39.550687 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:39.550689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:39.551113 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:39.551114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:39.551224 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:39.551699 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:39.551707 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:39.553175 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:39.554764 (+ 1589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:39.554770 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:39.554772 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:39.554991 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:39.555167 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:39.555168 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:39.555168 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:39.555265 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:39.555505 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:39.555510 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:39.555631 (+ 121us) client_negotiation.cc:769] Sending connection context
0928 14:43:39.555861 (+ 230us) client_negotiation.cc:240] Negotiation successful
0928 14:43:39.556082 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":99,"threads_started":1}
I20260928 14:43:39.556568 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 19649
I20260928 14:43:39.556686 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance
I20260928 14:43:39.556826 19778 heartbeater.cc:347] Connected to a master server at 127.9.33.126:33187
I20260928 14:43:39.556941 9349 external_mini_cluster.cc:1428] Setting key 7e77d00b218503b122261dced33c3ac8
I20260928 14:43:39.556949 19778 heartbeater.cc:464] Registering TS with master...
I20260928 14:43:39.557173 19778 heartbeater.cc:511] Master 127.9.33.126:33187 requested a full tablet report, sending...
I20260928 14:43:39.557778 19586 ts_manager.cc:194] Registered new tserver with Master: 7ba297d9e1bf42eead0ad93c2d5fb159 (127.9.33.65:41983)
I20260928 14:43:39.558485 19586 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.65@KRBTEST.COM'} at 127.9.33.65:35127
WARNING: no policy specified for kudu/127.9.33.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab.
Entry for principal kudu/127.9.33.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab.
I20260928 14:43:39.584271 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.66:0
--local_ip_for_outbound_sockets=127.9.33.66
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:33187
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:44105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:43:39.666671 19783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:39.666887 19783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:39.666913 19783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:39.668104 19783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:39.668150 19783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:39.668177 19783 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.66
I20260928 14:43:39.669605 19783 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:44105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:33187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.66
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:39.669817 19783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:39.669997 19783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:39.675309 19789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:39.675318 19788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:39.675328 19791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:39.675697 19783 server_base.cc:1034] running on GCE node
I20260928 14:43:39.675988 19783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:39.676221 19783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:39.677389 19783 hybrid_clock.cc:648] HybridClock initialized: now 1790606619677380 us; error 37 us; skew 500 ppm
Sep 28 14:43:39 dist-test-slave-pv0w krb5kdc[19542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606619, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:39.680202 19783 init.cc:377] Logged in from keytab as kudu/127.9.33.66@KRBTEST.COM (short username kudu)
I20260928 14:43:39.680675 19783 webserver.cc:460] Webserver started at http://127.9.33.66:32889/ using document root <none> and password file <none>
I20260928 14:43:39.680935 19783 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:39.681022 19783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:39.681180 19783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:39.682273 19783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/data/instance:
uuid: "4961e53035cd4a2cac08deef465274bd"
format_stamp: "Formatted at 2026-09-28 14:43:39 on dist-test-slave-pv0w"
server_key: "8cd3f9bfee8cc34b214dbcaff212a159"
server_key_iv: "92eb7a776c4e7c3f7066559f560d10fa"
server_key_version: "encryptionkey@0"
I20260928 14:43:39.682649 19783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance:
uuid: "4961e53035cd4a2cac08deef465274bd"
format_stamp: "Formatted at 2026-09-28 14:43:39 on dist-test-slave-pv0w"
server_key: "8cd3f9bfee8cc34b214dbcaff212a159"
server_key_iv: "92eb7a776c4e7c3f7066559f560d10fa"
server_key_version: "encryptionkey@0"
I20260928 14:43:39.684089 19783 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:43:39.684842 19798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:39.685020 19783 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 14:43:39.685127 19783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/wal
uuid: "4961e53035cd4a2cac08deef465274bd"
format_stamp: "Formatted at 2026-09-28 14:43:39 on dist-test-slave-pv0w"
server_key: "8cd3f9bfee8cc34b214dbcaff212a159"
server_key_iv: "92eb7a776c4e7c3f7066559f560d10fa"
server_key_version: "encryptionkey@0"
I20260928 14:43:39.685210 19783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:39.697098 19783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:39.699326 19783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:39.699445 19783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:39.699645 19783 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:43:39.700034 19783 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:43:39.700098 19783 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:39.700143 19783 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:43:39.700170 19783 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:39.706367 19783 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.66:36053
I20260928 14:43:39.706461 19911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.66:36053 every 8 connection(s)
I20260928 14:43:39.706840 19783 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
I20260928 14:43:39.709275 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 19783
I20260928 14:43:39.709388 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance
I20260928 14:43:39.709592 9349 external_mini_cluster.cc:1428] Setting key a6f9d395c4a6e9610b679685d8388b73
Sep 28 14:43:39 dist-test-slave-pv0w krb5kdc[19542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606619, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:43:39.718156 19626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:39.707641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:39045 (local address 127.9.33.126:33187)
0928 14:43:39.707699 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:39.707701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:39.707941 (+ 240us) server_negotiation.cc:407] Connection header received
0928 14:43:39.708442 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:39.708443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:39.708455 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:39.708508 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:39.709793 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:39.710334 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:39.712005 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:39.712252 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:39.712286 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:39.715042 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:39.715044 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:39.715044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:39.715052 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:39.716604 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:39.717135 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:39.717136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:39.717137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:39.717188 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:39.717490 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:39.717490 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:39.717490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:39.717597 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:39.717647 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:39.718019 (+ 372us) server_negotiation.cc:299] Negotiation successful
0928 14:43:39.718087 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260928 14:43:39.718497 19914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:39.707610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:33187 (local address 127.9.33.66:39045)
0928 14:43:39.707830 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:39.707867 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:39.708353 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:39.708574 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:39.708578 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:39.709299 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:39.709662 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:39.709671 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:39.710462 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:39.710464 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:39.711886 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:39.711890 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:39.712366 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:39.712366 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:39.712490 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:39.713080 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:39.713089 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:39.714919 (+ 1830us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:39.716693 (+ 1774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:39.716701 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:39.716704 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:39.717048 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:39.717252 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:39.717253 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:39.717253 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:39.717410 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:39.717663 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:39.717671 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:39.717854 (+ 183us) client_negotiation.cc:769] Sending connection context
0928 14:43:39.717997 (+ 143us) client_negotiation.cc:240] Negotiation successful
0928 14:43:39.718196 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":140,"threads_started":1}
WARNING: no policy specified for kudu/127.9.33.67@KRBTEST.COM; defaulting to no policy
I20260928 14:43:39.719079 19912 heartbeater.cc:347] Connected to a master server at 127.9.33.126:33187
I20260928 14:43:39.719203 19912 heartbeater.cc:464] Registering TS with master...
I20260928 14:43:39.719439 19912 heartbeater.cc:511] Master 127.9.33.126:33187 requested a full tablet report, sending...
I20260928 14:43:39.719893 19586 ts_manager.cc:194] Registered new tserver with Master: 4961e53035cd4a2cac08deef465274bd (127.9.33.66:36053)
I20260928 14:43:39.720337 19586 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.66@KRBTEST.COM'} at 127.9.33.66:39045
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab.
Entry for principal kudu/127.9.33.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab.
I20260928 14:43:39.739521 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.67:0
--local_ip_for_outbound_sockets=127.9.33.67
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:33187
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:44105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:43:39.823242 19917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:43:39.823537 19917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:43:39.823581 19917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:43:39.824975 19917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:43:39.825058 19917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:43:39.825107 19917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.67
I20260928 14:43:39.826715 19917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:44105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:33187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.67
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:43:39.827025 19917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:43:39.827281 19917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:43:39.832665 19923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:39.832665 19922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:43:39.832705 19925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:43:39.832945 19917 server_base.cc:1034] running on GCE node
I20260928 14:43:39.833170 19917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:43:39.833439 19917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:43:39.834625 19917 hybrid_clock.cc:648] HybridClock initialized: now 1790606619834612 us; error 32 us; skew 500 ppm
Sep 28 14:43:39 dist-test-slave-pv0w krb5kdc[19542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606619, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:43:39.837563 19917 init.cc:377] Logged in from keytab as kudu/127.9.33.67@KRBTEST.COM (short username kudu)
I20260928 14:43:39.838140 19917 webserver.cc:460] Webserver started at http://127.9.33.67:35011/ using document root <none> and password file <none>
I20260928 14:43:39.838466 19917 fs_manager.cc:359] Metadata directory not provided
I20260928 14:43:39.838546 19917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:43:39.838717 19917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:43:39.840019 19917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/data/instance:
uuid: "a8095a03c0894b219888a38c7b24d439"
format_stamp: "Formatted at 2026-09-28 14:43:39 on dist-test-slave-pv0w"
server_key: "3d80fc9c808ce89b05aee512836f0240"
server_key_iv: "ec8459d08e6c4a099383ea123f58080e"
server_key_version: "encryptionkey@0"
I20260928 14:43:39.840477 19917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance:
uuid: "a8095a03c0894b219888a38c7b24d439"
format_stamp: "Formatted at 2026-09-28 14:43:39 on dist-test-slave-pv0w"
server_key: "3d80fc9c808ce89b05aee512836f0240"
server_key_iv: "ec8459d08e6c4a099383ea123f58080e"
server_key_version: "encryptionkey@0"
I20260928 14:43:39.842391 19917 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260928 14:43:39.843253 19932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:39.843426 19917 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:43:39.843520 19917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/wal
uuid: "a8095a03c0894b219888a38c7b24d439"
format_stamp: "Formatted at 2026-09-28 14:43:39 on dist-test-slave-pv0w"
server_key: "3d80fc9c808ce89b05aee512836f0240"
server_key_iv: "ec8459d08e6c4a099383ea123f58080e"
server_key_version: "encryptionkey@0"
I20260928 14:43:39.843600 19917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:43:39.854048 19917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:43:39.856419 19917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:43:39.856547 19917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:43:39.856789 19917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:43:39.857155 19917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:43:39.857205 19917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:39.857230 19917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:43:39.857273 19917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:43:39.863790 19917 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.67:46647
I20260928 14:43:39.863847 20045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.67:46647 every 8 connection(s)
I20260928 14:43:39.864235 19917 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
I20260928 14:43:39.864789 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 19917
I20260928 14:43:39.864876 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance
I20260928 14:43:39.865091 9349 external_mini_cluster.cc:1428] Setting key 17aad6b6aaa6c2b12f84cf38a945286a
Sep 28 14:43:39 dist-test-slave-pv0w krb5kdc[19542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606619, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:43:39.874593 19626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:39.865133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:42449 (local address 127.9.33.126:33187)
0928 14:43:39.865190 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:39.865193 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:39.865391 (+ 198us) server_negotiation.cc:407] Connection header received
0928 14:43:39.865973 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:39.865974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:39.865988 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:39.866035 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:39.867054 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:39.867582 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:39.868715 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:39.868937 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:39.869018 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:39.871312 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:39.871314 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:39.871314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:39.871325 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:39.873023 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:39.873516 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:39.873517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:39.873518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:39.873584 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:39.873853 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:39.873855 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:39.873855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:39.873998 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:39.874079 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:39.874393 (+ 314us) server_negotiation.cc:299] Negotiation successful
0928 14:43:39.874483 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260928 14:43:39.874832 20048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:39.865112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:33187 (local address 127.9.33.67:42449)
0928 14:43:39.865274 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:39.865318 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:39.865820 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:39.866110 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:39.866115 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:39.866730 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:43:39.866947 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:39.866952 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:39.867691 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:39.867692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:39.868605 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:39.868608 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:39.869027 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:39.869028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:39.869178 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:39.869710 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:43:39.869720 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:43:39.871195 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:43:39.873149 (+ 1954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:39.873158 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:39.873161 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:39.873432 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:39.873655 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:43:39.873656 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:43:39.873656 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:43:39.873777 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:43:39.874069 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:43:39.874076 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:43:39.874238 (+ 162us) client_negotiation.cc:769] Sending connection context
0928 14:43:39.874356 (+ 118us) client_negotiation.cc:240] Negotiation successful
0928 14:43:39.874541 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":94,"threads_started":1}
I20260928 14:43:39.875418 20046 heartbeater.cc:347] Connected to a master server at 127.9.33.126:33187
I20260928 14:43:39.875591 20046 heartbeater.cc:464] Registering TS with master...
I20260928 14:43:39.875852 20046 heartbeater.cc:511] Master 127.9.33.126:33187 requested a full tablet report, sending...
I20260928 14:43:39.876298 19586 ts_manager.cc:194] Registered new tserver with Master: a8095a03c0894b219888a38c7b24d439 (127.9.33.67:46647)
I20260928 14:43:39.876739 19586 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.67@KRBTEST.COM'} at 127.9.33.67:42449
I20260928 14:43:39.876816 9349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:43:39.885100 19626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:39.878415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55682 (local address 127.9.33.126:33187)
0928 14:43:39.878457 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:39.878459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:39.878597 (+ 138us) server_negotiation.cc:407] Connection header received
0928 14:43:39.878623 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:39.878624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:39.878634 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:39.878679 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:43:39.879357 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:39.879985 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:39.880725 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:39.881012 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:39.881071 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:39.882238 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:43:39.882240 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:43:39.882240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:43:39.882247 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:43:39.883662 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:39.884105 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:39.884107 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:39.884107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:39.884172 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:43:39.884401 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:43:39.884402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:43:39.884402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:43:39.884567 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:43:39.884653 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:39.884757 (+ 104us) server_negotiation.cc:299] Negotiation successful
0928 14:43:39.884828 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 14:43:39.887063 19586 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55682:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:43:39.888895 19586 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:43:39.895793 20061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:39.891803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48598 (local address 127.9.33.66:36053)
0928 14:43:39.892069 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:39.892075 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:39.892087 (+ 12us) server_negotiation.cc:407] Connection header received
0928 14:43:39.892130 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:39.892132 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:39.892147 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:39.892446 (+ 299us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:43:39.892525 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:39.893213 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:39.894564 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:39.895318 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:39.895398 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:39.895473 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:39.895623 (+ 150us) server_negotiation.cc:299] Negotiation successful
0928 14:43:39.895666 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":146,"threads_started":1}
I20260928 14:43:39.895779 20060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:39.891581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:36053 (local address 127.0.0.1:48598)
0928 14:43:39.891905 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:39.891914 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:39.891986 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:39.892275 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:39.892278 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:39.892281 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:43:39.892363 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:39.892365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:39.893381 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:39.893382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:39.894482 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:39.894485 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:39.895411 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:39.895412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:39.895492 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:39.895516 (+ 24us) client_negotiation.cc:769] Sending connection context
0928 14:43:39.895592 (+ 76us) client_negotiation.cc:240] Negotiation successful
0928 14:43:39.895641 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"mutex_wait_us":4,"thread_start_us":50,"threads_started":1}
I20260928 14:43:39.895931 20062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:39.892017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46040 (local address 127.9.33.65:41983)
0928 14:43:39.892205 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:39.892209 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:39.892221 (+ 12us) server_negotiation.cc:407] Connection header received
0928 14:43:39.892294 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:39.892295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:39.892308 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:39.892404 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:43:39.892490 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:39.893206 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:39.894738 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:39.895434 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:39.895495 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:39.895544 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:39.895789 (+ 245us) server_negotiation.cc:299] Negotiation successful
0928 14:43:39.895840 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":87,"threads_started":1}
I20260928 14:43:39.896169 20058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:39.891799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:41983 (local address 127.0.0.1:46040)
0928 14:43:39.892067 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:39.892084 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:39.892212 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:39.892398 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:39.892399 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:39.892401 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:43:39.892444 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:39.892445 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:39.893315 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:39.893316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:39.894652 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:39.894655 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:39.895598 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:39.895599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:39.895665 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:39.895669 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:43:39.895723 (+ 54us) client_negotiation.cc:240] Negotiation successful
0928 14:43:39.896078 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":38,"thread_start_us":68,"threads_started":1}
I20260928 14:43:39.896777 19846 tablet_service.cc:1467] Processing CreateTablet for tablet 97abba8a97d24966beec7897be2f85b5 (DEFAULT_TABLE table=test-table [id=3f3b2ba0d54d418ba6f542961f98769c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:43:39.896926 20057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:39.891581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:46647 (local address 127.0.0.1:46740)
0928 14:43:39.891834 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:39.891880 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:39.891957 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:39.893609 (+ 1652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:39.893610 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:39.893613 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:43:39.893651 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:39.893653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:39.894875 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:39.894876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:39.895692 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:39.895694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:39.896684 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:39.896685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:39.896749 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:39.896755 (+ 6us) client_negotiation.cc:769] Sending connection context
0928 14:43:39.896809 (+ 54us) client_negotiation.cc:240] Negotiation successful
0928 14:43:39.896857 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":90,"threads_started":1}
I20260928 14:43:39.896938 20059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:39.891676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46740 (local address 127.9.33.67:46647)
0928 14:43:39.893446 (+ 1770us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:39.893450 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:39.893460 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:43:39.893495 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:39.893497 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:39.893512 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:39.893593 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:43:39.893720 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:39.894762 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:39.895829 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:39.896591 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:39.896683 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:39.896740 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:39.896821 (+ 81us) server_negotiation.cc:299] Negotiation successful
0928 14:43:39.896860 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1725,"thread_start_us":129,"threads_started":1}
I20260928 14:43:39.897154 19846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97abba8a97d24966beec7897be2f85b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:39.899909 20063 tablet_bootstrap.cc:492] T 97abba8a97d24966beec7897be2f85b5 P 4961e53035cd4a2cac08deef465274bd: Bootstrap starting.
I20260928 14:43:39.900626 20063 tablet_bootstrap.cc:654] T 97abba8a97d24966beec7897be2f85b5 P 4961e53035cd4a2cac08deef465274bd: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:39.900519 19712 tablet_service.cc:1467] Processing CreateTablet for tablet 97abba8a97d24966beec7897be2f85b5 (DEFAULT_TABLE table=test-table [id=3f3b2ba0d54d418ba6f542961f98769c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:43:39.900691 19980 tablet_service.cc:1467] Processing CreateTablet for tablet 97abba8a97d24966beec7897be2f85b5 (DEFAULT_TABLE table=test-table [id=3f3b2ba0d54d418ba6f542961f98769c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:43:39.901012 19712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97abba8a97d24966beec7897be2f85b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:39.901069 19980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97abba8a97d24966beec7897be2f85b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:43:39.901224 20063 log.cc:824] T 97abba8a97d24966beec7897be2f85b5 P 4961e53035cd4a2cac08deef465274bd: Log is configured to *not* fsync() on all Append() calls
I20260928 14:43:39.902081 20063 tablet_bootstrap.cc:492] T 97abba8a97d24966beec7897be2f85b5 P 4961e53035cd4a2cac08deef465274bd: No bootstrap required, opened a new log
I20260928 14:43:39.902154 20063 ts_tablet_manager.cc:1397] T 97abba8a97d24966beec7897be2f85b5 P 4961e53035cd4a2cac08deef465274bd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:43:39.904072 20065 tablet_bootstrap.cc:492] T 97abba8a97d24966beec7897be2f85b5 P 7ba297d9e1bf42eead0ad93c2d5fb159: Bootstrap starting.
I20260928 14:43:39.904281 20063 raft_consensus.cc:348] T 97abba8a97d24966beec7897be2f85b5 P 4961e53035cd4a2cac08deef465274bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8095a03c0894b219888a38c7b24d439" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 46647 } } peers { permanent_uuid: "4961e53035cd4a2cac08deef465274bd" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36053 } } peers { permanent_uuid: "7ba297d9e1bf42eead0ad93c2d5fb159" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41983 } }
I20260928 14:43:39.904481 20063 raft_consensus.cc:374] T 97abba8a97d24966beec7897be2f85b5 P 4961e53035cd4a2cac08deef465274bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:39.904532 20063 raft_consensus.cc:729] T 97abba8a97d24966beec7897be2f85b5 P 4961e53035cd4a2cac08deef465274bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4961e53035cd4a2cac08deef465274bd, State: Initialized, Role: FOLLOWER
I20260928 14:43:39.904608 20066 tablet_bootstrap.cc:492] T 97abba8a97d24966beec7897be2f85b5 P a8095a03c0894b219888a38c7b24d439: Bootstrap starting.
I20260928 14:43:39.904675 20063 consensus_queue.cc:260] T 97abba8a97d24966beec7897be2f85b5 P 4961e53035cd4a2cac08deef465274bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8095a03c0894b219888a38c7b24d439" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 46647 } } peers { permanent_uuid: "4961e53035cd4a2cac08deef465274bd" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36053 } } peers { permanent_uuid: "7ba297d9e1bf42eead0ad93c2d5fb159" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41983 } }
I20260928 14:43:39.904999 20063 ts_tablet_manager.cc:1428] T 97abba8a97d24966beec7897be2f85b5 P 4961e53035cd4a2cac08deef465274bd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:43:39.905138 19912 heartbeater.cc:503] Master 127.9.33.126:33187 was elected leader, sending a full tablet report...
I20260928 14:43:39.905265 20066 tablet_bootstrap.cc:654] T 97abba8a97d24966beec7897be2f85b5 P a8095a03c0894b219888a38c7b24d439: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:39.905265 20065 tablet_bootstrap.cc:654] T 97abba8a97d24966beec7897be2f85b5 P 7ba297d9e1bf42eead0ad93c2d5fb159: Neither blocks nor log segments found. Creating new log.
I20260928 14:43:39.905887 20066 log.cc:824] T 97abba8a97d24966beec7897be2f85b5 P a8095a03c0894b219888a38c7b24d439: Log is configured to *not* fsync() on all Append() calls
I20260928 14:43:39.905890 20065 log.cc:824] T 97abba8a97d24966beec7897be2f85b5 P 7ba297d9e1bf42eead0ad93c2d5fb159: Log is configured to *not* fsync() on all Append() calls
I20260928 14:43:39.906690 20066 tablet_bootstrap.cc:492] T 97abba8a97d24966beec7897be2f85b5 P a8095a03c0894b219888a38c7b24d439: No bootstrap required, opened a new log
I20260928 14:43:39.906754 20065 tablet_bootstrap.cc:492] T 97abba8a97d24966beec7897be2f85b5 P 7ba297d9e1bf42eead0ad93c2d5fb159: No bootstrap required, opened a new log
I20260928 14:43:39.906778 20066 ts_tablet_manager.cc:1397] T 97abba8a97d24966beec7897be2f85b5 P a8095a03c0894b219888a38c7b24d439: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:43:39.906847 20065 ts_tablet_manager.cc:1397] T 97abba8a97d24966beec7897be2f85b5 P 7ba297d9e1bf42eead0ad93c2d5fb159: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260928 14:43:39.908428 20066 raft_consensus.cc:348] T 97abba8a97d24966beec7897be2f85b5 P a8095a03c0894b219888a38c7b24d439 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8095a03c0894b219888a38c7b24d439" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 46647 } } peers { permanent_uuid: "4961e53035cd4a2cac08deef465274bd" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36053 } } peers { permanent_uuid: "7ba297d9e1bf42eead0ad93c2d5fb159" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41983 } }
I20260928 14:43:39.908587 20066 raft_consensus.cc:374] T 97abba8a97d24966beec7897be2f85b5 P a8095a03c0894b219888a38c7b24d439 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:39.908634 20066 raft_consensus.cc:729] T 97abba8a97d24966beec7897be2f85b5 P a8095a03c0894b219888a38c7b24d439 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8095a03c0894b219888a38c7b24d439, State: Initialized, Role: FOLLOWER
I20260928 14:43:39.908546 20065 raft_consensus.cc:348] T 97abba8a97d24966beec7897be2f85b5 P 7ba297d9e1bf42eead0ad93c2d5fb159 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8095a03c0894b219888a38c7b24d439" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 46647 } } peers { permanent_uuid: "4961e53035cd4a2cac08deef465274bd" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36053 } } peers { permanent_uuid: "7ba297d9e1bf42eead0ad93c2d5fb159" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41983 } }
I20260928 14:43:39.908730 20065 raft_consensus.cc:374] T 97abba8a97d24966beec7897be2f85b5 P 7ba297d9e1bf42eead0ad93c2d5fb159 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:43:39.908782 20065 raft_consensus.cc:729] T 97abba8a97d24966beec7897be2f85b5 P 7ba297d9e1bf42eead0ad93c2d5fb159 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ba297d9e1bf42eead0ad93c2d5fb159, State: Initialized, Role: FOLLOWER
I20260928 14:43:39.908757 20066 consensus_queue.cc:260] T 97abba8a97d24966beec7897be2f85b5 P a8095a03c0894b219888a38c7b24d439 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8095a03c0894b219888a38c7b24d439" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 46647 } } peers { permanent_uuid: "4961e53035cd4a2cac08deef465274bd" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36053 } } peers { permanent_uuid: "7ba297d9e1bf42eead0ad93c2d5fb159" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41983 } }
I20260928 14:43:39.908905 20065 consensus_queue.cc:260] T 97abba8a97d24966beec7897be2f85b5 P 7ba297d9e1bf42eead0ad93c2d5fb159 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8095a03c0894b219888a38c7b24d439" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 46647 } } peers { permanent_uuid: "4961e53035cd4a2cac08deef465274bd" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36053 } } peers { permanent_uuid: "7ba297d9e1bf42eead0ad93c2d5fb159" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41983 } }
I20260928 14:43:39.909029 20046 heartbeater.cc:503] Master 127.9.33.126:33187 was elected leader, sending a full tablet report...
I20260928 14:43:39.909040 20066 ts_tablet_manager.cc:1428] T 97abba8a97d24966beec7897be2f85b5 P a8095a03c0894b219888a38c7b24d439: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260928 14:43:39.909197 20065 ts_tablet_manager.cc:1428] T 97abba8a97d24966beec7897be2f85b5 P 7ba297d9e1bf42eead0ad93c2d5fb159: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260928 14:43:39.909276 19778 heartbeater.cc:503] Master 127.9.33.126:33187 was elected leader, sending a full tablet report...
W20260928 14:43:39.957504 19913 tablet.cc:2369] T 97abba8a97d24966beec7897be2f85b5 P 4961e53035cd4a2cac08deef465274bd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:43:40.047137 19779 tablet.cc:2369] T 97abba8a97d24966beec7897be2f85b5 P 7ba297d9e1bf42eead0ad93c2d5fb159: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:43:40.114742 20071 raft_consensus.cc:482] T 97abba8a97d24966beec7897be2f85b5 P 7ba297d9e1bf42eead0ad93c2d5fb159 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:43:40.114945 20071 raft_consensus.cc:504] T 97abba8a97d24966beec7897be2f85b5 P 7ba297d9e1bf42eead0ad93c2d5fb159 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8095a03c0894b219888a38c7b24d439" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 46647 } } peers { permanent_uuid: "4961e53035cd4a2cac08deef465274bd" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36053 } } peers { permanent_uuid: "7ba297d9e1bf42eead0ad93c2d5fb159" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41983 } }
W20260928 14:43:40.115033 20047 tablet.cc:2369] T 97abba8a97d24966beec7897be2f85b5 P a8095a03c0894b219888a38c7b24d439: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:43:40.115358 20071 leader_election.cc:288] T 97abba8a97d24966beec7897be2f85b5 P 7ba297d9e1bf42eead0ad93c2d5fb159 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a8095a03c0894b219888a38c7b24d439 (127.9.33.67:46647), 4961e53035cd4a2cac08deef465274bd (127.9.33.66:36053)
I20260928 14:43:40.118470 20059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:40.115634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:48227 (local address 127.9.33.67:46647)
0928 14:43:40.115697 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:40.115700 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:40.115766 (+ 66us) server_negotiation.cc:407] Connection header received
0928 14:43:40.115856 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:40.115857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:40.115869 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:40.115933 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:43:40.116113 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:40.116649 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:40.117714 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:40.118102 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:40.118190 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:40.118203 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:40.118350 (+ 147us) server_negotiation.cc:299] Negotiation successful
0928 14:43:40.118393 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260928 14:43:40.118484 20061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:40.115681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:41027 (local address 127.9.33.66:36053)
0928 14:43:40.115750 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:40.115752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:40.115827 (+ 75us) server_negotiation.cc:407] Connection header received
0928 14:43:40.115920 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:40.115921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:40.115934 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:40.115992 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:43:40.116194 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:40.116661 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:40.117712 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:40.118104 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:40.118190 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:40.118202 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:40.118376 (+ 174us) server_negotiation.cc:299] Negotiation successful
0928 14:43:40.118415 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260928 14:43:40.118505 20073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:40.115586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:36053 (local address 127.9.33.65:41027)
0928 14:43:40.115767 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:40.115778 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:40.115844 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:40.116007 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:40.116008 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:40.116012 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:43:40.116117 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:40.116119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:40.116766 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:40.116767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:40.117615 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:40.117617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:40.118197 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:40.118198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:40.118282 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:40.118286 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:43:40.118358 (+ 72us) client_negotiation.cc:240] Negotiation successful
0928 14:43:40.118406 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":13,"thread_start_us":68,"threads_started":1}
I20260928 14:43:40.118513 20072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:40.115531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:46647 (local address 127.9.33.65:48227)
0928 14:43:40.115681 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:40.115693 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:43:40.115783 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:43:40.115945 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:43:40.115946 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:43:40.115948 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:43:40.116030 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:40.116032 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:40.116736 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:40.116737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:40.117613 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:43:40.117615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:43:40.118193 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:43:40.118195 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:43:40.118268 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:40.118272 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:43:40.118325 (+ 53us) client_negotiation.cc:240] Negotiation successful
0928 14:43:40.118405 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":28,"thread_start_us":70,"threads_started":1}
I20260928 14:43:40.118816 19866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97abba8a97d24966beec7897be2f85b5" candidate_uuid: "7ba297d9e1bf42eead0ad93c2d5fb159" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4961e53035cd4a2cac08deef465274bd" is_pre_election: true
I20260928 14:43:40.119015 19866 raft_consensus.cc:2454] T 97abba8a97d24966beec7897be2f85b5 P 4961e53035cd4a2cac08deef465274bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ba297d9e1bf42eead0ad93c2d5fb159 in term 0.
I20260928 14:43:40.119009 20000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97abba8a97d24966beec7897be2f85b5" candidate_uuid: "7ba297d9e1bf42eead0ad93c2d5fb159" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8095a03c0894b219888a38c7b24d439" is_pre_election: true
I20260928 14:43:40.119154 20000 raft_consensus.cc:2454] T 97abba8a97d24966beec7897be2f85b5 P a8095a03c0894b219888a38c7b24d439 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ba297d9e1bf42eead0ad93c2d5fb159 in term 0.
I20260928 14:43:40.119298 19666 leader_election.cc:302] T 97abba8a97d24966beec7897be2f85b5 P 7ba297d9e1bf42eead0ad93c2d5fb159 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4961e53035cd4a2cac08deef465274bd, 7ba297d9e1bf42eead0ad93c2d5fb159; no voters:
I20260928 14:43:40.119419 20071 raft_consensus.cc:2781] T 97abba8a97d24966beec7897be2f85b5 P 7ba297d9e1bf42eead0ad93c2d5fb159 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:43:40.119479 20071 raft_consensus.cc:482] T 97abba8a97d24966beec7897be2f85b5 P 7ba297d9e1bf42eead0ad93c2d5fb159 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:43:40.119505 20071 raft_consensus.cc:3037] T 97abba8a97d24966beec7897be2f85b5 P 7ba297d9e1bf42eead0ad93c2d5fb159 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:40.120383 20071 raft_consensus.cc:504] T 97abba8a97d24966beec7897be2f85b5 P 7ba297d9e1bf42eead0ad93c2d5fb159 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8095a03c0894b219888a38c7b24d439" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 46647 } } peers { permanent_uuid: "4961e53035cd4a2cac08deef465274bd" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36053 } } peers { permanent_uuid: "7ba297d9e1bf42eead0ad93c2d5fb159" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41983 } }
I20260928 14:43:40.120568 20071 leader_election.cc:288] T 97abba8a97d24966beec7897be2f85b5 P 7ba297d9e1bf42eead0ad93c2d5fb159 [CANDIDATE]: Term 1 election: Requested vote from peers a8095a03c0894b219888a38c7b24d439 (127.9.33.67:46647), 4961e53035cd4a2cac08deef465274bd (127.9.33.66:36053)
I20260928 14:43:40.120787 19866 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97abba8a97d24966beec7897be2f85b5" candidate_uuid: "7ba297d9e1bf42eead0ad93c2d5fb159" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4961e53035cd4a2cac08deef465274bd"
I20260928 14:43:40.120788 20000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97abba8a97d24966beec7897be2f85b5" candidate_uuid: "7ba297d9e1bf42eead0ad93c2d5fb159" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8095a03c0894b219888a38c7b24d439"
I20260928 14:43:40.120882 19866 raft_consensus.cc:3037] T 97abba8a97d24966beec7897be2f85b5 P 4961e53035cd4a2cac08deef465274bd [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:40.120882 20000 raft_consensus.cc:3037] T 97abba8a97d24966beec7897be2f85b5 P a8095a03c0894b219888a38c7b24d439 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:43:40.121937 20000 raft_consensus.cc:2454] T 97abba8a97d24966beec7897be2f85b5 P a8095a03c0894b219888a38c7b24d439 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ba297d9e1bf42eead0ad93c2d5fb159 in term 1.
I20260928 14:43:40.121937 19866 raft_consensus.cc:2454] T 97abba8a97d24966beec7897be2f85b5 P 4961e53035cd4a2cac08deef465274bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ba297d9e1bf42eead0ad93c2d5fb159 in term 1.
I20260928 14:43:40.122152 19666 leader_election.cc:302] T 97abba8a97d24966beec7897be2f85b5 P 7ba297d9e1bf42eead0ad93c2d5fb159 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4961e53035cd4a2cac08deef465274bd, 7ba297d9e1bf42eead0ad93c2d5fb159; no voters:
I20260928 14:43:40.122256 20071 raft_consensus.cc:2781] T 97abba8a97d24966beec7897be2f85b5 P 7ba297d9e1bf42eead0ad93c2d5fb159 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:43:40.122431 20071 raft_consensus.cc:686] T 97abba8a97d24966beec7897be2f85b5 P 7ba297d9e1bf42eead0ad93c2d5fb159 [term 1 LEADER]: Becoming Leader. State: Replica: 7ba297d9e1bf42eead0ad93c2d5fb159, State: Running, Role: LEADER
I20260928 14:43:40.122551 20071 consensus_queue.cc:237] T 97abba8a97d24966beec7897be2f85b5 P 7ba297d9e1bf42eead0ad93c2d5fb159 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8095a03c0894b219888a38c7b24d439" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 46647 } } peers { permanent_uuid: "4961e53035cd4a2cac08deef465274bd" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36053 } } peers { permanent_uuid: "7ba297d9e1bf42eead0ad93c2d5fb159" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41983 } }
I20260928 14:43:40.123476 19586 catalog_manager.cc:5495] T 97abba8a97d24966beec7897be2f85b5 P 7ba297d9e1bf42eead0ad93c2d5fb159 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ba297d9e1bf42eead0ad93c2d5fb159 (127.9.33.65). New cstate: current_term: 1 leader_uuid: "7ba297d9e1bf42eead0ad93c2d5fb159" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a8095a03c0894b219888a38c7b24d439" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 46647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4961e53035cd4a2cac08deef465274bd" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ba297d9e1bf42eead0ad93c2d5fb159" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41983 } health_report { overall_health: HEALTHY } } }
I20260928 14:43:40.184154 20062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:40.181645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46052 (local address 127.9.33.65:41983)
0928 14:43:40.181696 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 14:43:40.181698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:43:40.181709 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:43:40.181758 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:43:40.181759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:43:40.181790 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:43:40.181875 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:43:40.182018 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:40.182508 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:40.183295 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:43:40.183499 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:43:40.183580 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:43:40.183783 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:43:40.183898 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:43:40.184009 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0928 14:43:40.184027 (+ 18us) server_negotiation.cc:299] Negotiation successful
0928 14:43:40.184050 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260928 14:43:40.186283 19866 raft_consensus.cc:1261] T 97abba8a97d24966beec7897be2f85b5 P 4961e53035cd4a2cac08deef465274bd [term 1 FOLLOWER]: Refusing update from remote peer 7ba297d9e1bf42eead0ad93c2d5fb159: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:43:40.186286 20000 raft_consensus.cc:1261] T 97abba8a97d24966beec7897be2f85b5 P a8095a03c0894b219888a38c7b24d439 [term 1 FOLLOWER]: Refusing update from remote peer 7ba297d9e1bf42eead0ad93c2d5fb159: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:43:40.186645 20071 consensus_queue.cc:1035] T 97abba8a97d24966beec7897be2f85b5 P 7ba297d9e1bf42eead0ad93c2d5fb159 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4961e53035cd4a2cac08deef465274bd" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 36053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:43:40.186798 20074 consensus_queue.cc:1035] T 97abba8a97d24966beec7897be2f85b5 P 7ba297d9e1bf42eead0ad93c2d5fb159 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8095a03c0894b219888a38c7b24d439" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 46647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:43:40.190398 20078 mvcc.cc:205] Tried to move back new op lower bound from 7334324716280070144 to 7334324716022562816. Current Snapshot: MvccSnapshot[applied={T|T < 7334324716280070144}]
I20260928 14:43:40.190510 20079 mvcc.cc:205] Tried to move back new op lower bound from 7334324716280070144 to 7334324716022562816. Current Snapshot: MvccSnapshot[applied={T|T < 7334324716280070144 or (T in {7334324716280070144})}]
I20260928 14:43:40.192011 19587 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55682:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:43:40.192112 19587 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55682:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:43:40.192946 19587 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 130bf81aec574e71a6751916b68d53bb: Sending DeleteTablet for 3 replicas of tablet 97abba8a97d24966beec7897be2f85b5
I20260928 14:43:40.193321 19980 tablet_service.cc:1514] Processing DeleteTablet for tablet 97abba8a97d24966beec7897be2f85b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:43:40 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:46740
I20260928 14:43:40.193358 19846 tablet_service.cc:1514] Processing DeleteTablet for tablet 97abba8a97d24966beec7897be2f85b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:43:40 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:48598
I20260928 14:43:40.193574 20087 tablet_replica.cc:330] T 97abba8a97d24966beec7897be2f85b5 P 4961e53035cd4a2cac08deef465274bd: stopping tablet replica
I20260928 14:43:40.193691 20087 raft_consensus.cc:2229] T 97abba8a97d24966beec7897be2f85b5 P 4961e53035cd4a2cac08deef465274bd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:43:40.193775 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 19649
I20260928 14:43:40.193802 20087 raft_consensus.cc:2258] T 97abba8a97d24966beec7897be2f85b5 P 4961e53035cd4a2cac08deef465274bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:43:40.193768 19712 tablet_service.cc:1514] Processing DeleteTablet for tablet 97abba8a97d24966beec7897be2f85b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:43:40 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:46040
I20260928 14:43:40.194736 20087 ts_tablet_manager.cc:1905] T 97abba8a97d24966beec7897be2f85b5 P 4961e53035cd4a2cac08deef465274bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:43:40.195354 20088 tablet_replica.cc:330] T 97abba8a97d24966beec7897be2f85b5 P a8095a03c0894b219888a38c7b24d439: stopping tablet replica
I20260928 14:43:40.195482 20088 raft_consensus.cc:2229] T 97abba8a97d24966beec7897be2f85b5 P a8095a03c0894b219888a38c7b24d439 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:43:40.195600 20088 raft_consensus.cc:2258] T 97abba8a97d24966beec7897be2f85b5 P a8095a03c0894b219888a38c7b24d439 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:43:40.196161 20088 ts_tablet_manager.cc:1905] T 97abba8a97d24966beec7897be2f85b5 P a8095a03c0894b219888a38c7b24d439: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:43:40.196856 20087 ts_tablet_manager.cc:1918] T 97abba8a97d24966beec7897be2f85b5 P 4961e53035cd4a2cac08deef465274bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:43:40.196941 20087 log.cc:1196] T 97abba8a97d24966beec7897be2f85b5 P 4961e53035cd4a2cac08deef465274bd: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-1/wal/wals/97abba8a97d24966beec7897be2f85b5
I20260928 14:43:40.197211 20087 ts_tablet_manager.cc:1939] T 97abba8a97d24966beec7897be2f85b5 P 4961e53035cd4a2cac08deef465274bd: Deleting consensus metadata
I20260928 14:43:40.197693 20088 ts_tablet_manager.cc:1918] T 97abba8a97d24966beec7897be2f85b5 P a8095a03c0894b219888a38c7b24d439: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:43:40.197670 19572 catalog_manager.cc:4841] TS 4961e53035cd4a2cac08deef465274bd (127.9.33.66:36053): tablet 97abba8a97d24966beec7897be2f85b5 (table test-table [id=3f3b2ba0d54d418ba6f542961f98769c]) successfully deleted
I20260928 14:43:40.197763 20088 log.cc:1196] T 97abba8a97d24966beec7897be2f85b5 P a8095a03c0894b219888a38c7b24d439: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790606561545179-9349-0/minicluster-data/ts-2/wal/wals/97abba8a97d24966beec7897be2f85b5
I20260928 14:43:40.198247 20088 ts_tablet_manager.cc:1939] T 97abba8a97d24966beec7897be2f85b5 P a8095a03c0894b219888a38c7b24d439: Deleting consensus metadata
I20260928 14:43:40.198728 19573 catalog_manager.cc:4841] TS a8095a03c0894b219888a38c7b24d439 (127.9.33.67:46647): tablet 97abba8a97d24966beec7897be2f85b5 (table test-table [id=3f3b2ba0d54d418ba6f542961f98769c]) successfully deleted
W20260928 14:43:40.200009 19572 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.33.65:41983 (error 108)
I20260928 14:43:40.200264 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 19783
I20260928 14:43:40.201349 20057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:43:40.200495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:41983 (local address 127.0.0.1:46062)
0928 14:43:40.201271 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0928 14:43:40.201310 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.65:41983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":754}
W20260928 14:43:40.201488 19572 catalog_manager.cc:4584] TS 7ba297d9e1bf42eead0ad93c2d5fb159 (127.9.33.65:41983): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 97abba8a97d24966beec7897be2f85b5: Network error: Client connection negotiation failed: client connection to 127.9.33.65:41983: connect: Connection refused (error 111)
I20260928 14:43:40.206342 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 19917
I20260928 14:43:40.210843 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 19555
2026-09-28T14:43:40Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2182 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:43:40 dist-test-slave-pv0w krb5kdc[20092](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:43:40 dist-test-slave-pv0w krb5kdc[20092](info): set up 2 sockets
Sep 28 14:43:40 dist-test-slave-pv0w krb5kdc[20092](info): commencing operation
krb5kdc: starting...
W20260928 14:43:41.261641 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:43:41 dist-test-slave-pv0w krb5kdc[20092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:43:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:43:41Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.9.33.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.9.33.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.9.33.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/rangeradmin_127.9.33.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.9.33.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/rangeradmin_127.9.33.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.9.33.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.9.33.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.9.33.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/rangerlookup_127.9.33.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.9.33.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/rangerlookup_127.9.33.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.33.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.9.33.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.33.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/HTTP_127.9.33.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.33.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/HTTP_127.9.33.84@KRBTEST.COM.keytab.
I20260928 14:43:41.443559 9349 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task4TnxN9/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/postgres -l logfile start
2026-09-28 14:43:44.189 UTC [20122] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-28 14:43:44.189 UTC [20122] LOG: listening on IPv4 address "127.9.33.84", port 44947
2026-09-28 14:43:44.193 UTC [20122] LOG: listening on Unix socket "/tmp/.s.PGSQL.44947"
2026-09-28 14:43:44.215 UTC [20125] LOG: database system was shut down at 2026-09-28 14:43:42 UTC
2026-09-28 14:43:44.220 UTC [20122] LOG: database system is ready to accept connections
I20260928 14:43:45.169392 9349 mini_postgres.cc:96] Postgres bound to 44947
2026-09-28 14:43:45.175 UTC [20133] FATAL: database "slave" does not exist
127.9.33.84:44947 - accepting connections
I20260928 14:43:45.176237 9349 mini_ranger.cc:161] Starting Ranger...
I20260928 14:43:45.191268 9349 mini_ranger.cc:84] Created miniranger Postgres user
I20260928 14:43:45.723394 9349 mini_ranger.cc:87] Created ranger Postgres database
I20260928 14:43:45.723485 9349 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ranger-admin
2026-09-28 14:43:46,089 [I] DB FLAVOR :POSTGRES
2026-09-28 14:43:46,089 [I] --------- Verifying Ranger DB connection ---------
2026-09-28 14:43:46,090 [I] Checking connection..
2026-09-28 14:43:46,090 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:44947/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-28 14:43:46,364 [I] Checking connection passed.
2026-09-28 14:43:46,364 [I] --------- Verifying version history table ---------
2026-09-28 14:43:46,365 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:44947/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 14:43:46,632 [I] Table x_db_version_h does not exist in database ranger
2026-09-28 14:43:46,632 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:44947/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 14:43:46,877 [I] Table x_db_version_h does not exist in database ranger
2026-09-28 14:43:46,877 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-28 14:43:46,877 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:44947/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-28 14:43:47.118 UTC [20197] WARNING: there is no transaction in progress
2026-09-28 14:43:47,125 [I] create_dbversion_catalog.sql file imported successfully
2026-09-28 14:43:47,126 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:44947/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 14:43:47,396 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-28 14:43:47,396 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-28 14:43:47,396 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:44947/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-28 14:43:47,662 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:44947/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-28 14:43:47,904 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:44947/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-pv0w.c.gcp-upstream.internal','N') ;"
2026-09-28 14:43:48,160 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:44947/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-28 14:43:48,430 [I] Table x_portal_user does not exist in database ranger
2026-09-28 14:43:48,430 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:44947/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-28 14:43:48,707 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-28 14:43:48,707 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:44947/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-28 14:43:48,951 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:44947/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-28 14:43:49,199 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-28 14:43:49,199 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:44947/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-28 14:43:50.266 UTC [20305] WARNING: there is no transaction in progress
2026-09-28 14:43:50.285 UTC [20305] WARNING: there is no transaction in progress
2026-09-28 14:43:50.303 UTC [20305] WARNING: there is no transaction in progress
2026-09-28 14:43:50.322 UTC [20305] WARNING: there is no transaction in progress
2026-09-28 14:43:50.344 UTC [20305] WARNING: there is no transaction in progress
2026-09-28 14:43:50.364 UTC [20305] WARNING: there is no transaction in progress
2026-09-28 14:43:50.424 UTC [20305] WARNING: there is no transaction in progress
2026-09-28 14:43:50.431 UTC [20305] WARNING: there is no transaction in progress
2026-09-28 14:43:50.445 UTC [20305] WARNING: there is no transaction in progress
2026-09-28 14:43:50.456 UTC [20305] WARNING: there is no transaction in progress
2026-09-28 14:43:50.466 UTC [20305] WARNING: there is no transaction in progress
2026-09-28 14:43:50.480 UTC [20305] WARNING: there is no transaction in progress
2026-09-28 14:43:50.491 UTC [20305] WARNING: there is no transaction in progress
2026-09-28 14:43:50.499 UTC [20305] WARNING: there is no transaction in progress
2026-09-28 14:43:50.507 UTC [20305] WARNING: there is no transaction in progress
2026-09-28 14:43:50.514 UTC [20305] WARNING: there is no transaction in progress
2026-09-28 14:43:51,209 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-28 14:43:51,209 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:44947/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-28 14:43:51,472 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-28 14:43:51,472 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:44947/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-28 14:43:51,718 [I] Table x_portal_user already exists in database 'ranger'
2026-09-28 14:43:51,718 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:44947/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-28 14:43:51,961 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-28 14:43:51,962 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:44947/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-28 14:43:52,218 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:44947/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-28 14:43:52,460 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:44947/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-pv0w.c.gcp-upstream.internal';"
2026-09-28 14:43:52,699 [I] CORE_DB_SCHEMA import status has been updated
2026-09-28 14:43:52,699 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:44947/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-28 14:43:52,956 [I] DB_PATCHES have already been applied
I20260928 14:43:52.962375 9349 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ranger-admin:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task4TnxN9/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260928 14:43:52.962441 9349 mini_ranger.cc:193] Using host: 127.9.33.84
I20260928 14:43:52.965597 9349 mini_ranger.cc:227] Ranger admin URL: http://127.9.33.84:46613
Sep 28, 2026 2:43:53 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 28, 2026 2:43:53 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 28, 2026 2:43:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ranger-admin/ews/webapp
Sep 28, 2026 2:43:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 28, 2026 2:43:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 28, 2026 2:43:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ranger-admin/ews/webapp].
Sep 28, 2026 2:43:54 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260928 14:43:55.260479 9349 mini_ranger.cc:160] Time spent starting Ranger: real 10.084s user 0.000s sys 0.011s
Sep 28, 2026 2:44:16 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 28, 2026 2:44:16 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 28, 2026 2:44:16 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 28, 2026 2:44:16 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 28, 2026 2:44:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 28, 2026 2:44:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 28, 2026 2:44:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 28, 2026 2:44:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 28, 2026 2:44:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 28, 2026 2:44:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 28, 2026 2:44:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 28, 2026 2:44:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 28, 2026 2:44:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 28, 2026 2:44:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 28, 2026 2:44:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 28, 2026 2:44:16 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 28, 2026 2:44:16 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 28, 2026 2:44:17 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260928 14:44:18.399003 9349 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.9.33.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.9.33.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.9.33.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/rangerkms_127.9.33.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.9.33.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/rangerkms_127.9.33.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.33.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.9.33.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.33.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/HTTP_127.9.33.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.33.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/HTTP_127.9.33.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 28 14:44:18 dist-test-slave-pv0w krb5kdc[20092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606658, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260928 14:44:18.485297 9349 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260928 14:44:18.494865 9349 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260928 14:44:19.391616 9349 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260928 14:44:19.391733 9349 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ranger-kms
I20260928 14:44:19.391759 9349 mini_ranger_kms.cc:227] Using postgres at 127.9.33.84:44947
2026-09-28 14:44:19,832 [I] DB FLAVOR :POSTGRES
2026-09-28 14:44:19,832 [I] --------- Verifying Ranger DB connection ---------
2026-09-28 14:44:19,833 [I] Checking connection
2026-09-28 14:44:19,833 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:44947/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-28 14:44:20,177 [I] connection success
2026-09-28 14:44:20,177 [I] --------- Verifying Ranger DB tables ---------
2026-09-28 14:44:20,178 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-28 14:44:20,178 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:44947/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-28 14:44:20,468 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-28 14:44:20,469 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-28 14:44:20,469 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-28 14:44:20,469 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:44947/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-28 14:44:20,756 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260928 14:44:21.308518 9349 mini_ranger_kms.cc:310] Created kms service
I20260928 14:44:21.498710 9349 mini_ranger_kms.cc:326] Created kudu user
I20260928 14:44:21.545554 9349 mini_ranger_kms.cc:343] Created rangerkms user
Sep 28 14:44:21 dist-test-slave-pv0w krb5kdc[20092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606658, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.9.33.84@KRBTEST.COM
I20260928 14:44:22.010522 9349 mini_ranger_kms.cc:384] Added ranger policy
I20260928 14:44:22.010663 9349 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ranger-kms:/tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260928 14:44:22.010690 9349 mini_ranger_kms.cc:240] Using host: 127.9.33.84
I20260928 14:44:22.016506 9349 mini_ranger_kms.cc:276] Ranger KMS PID: 20487
I20260928 14:44:22.016646 9349 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.9.33.84:35471
Sep 28, 2026 2:44:23 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 28, 2026 2:44:23 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 28, 2026 2:44:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 28, 2026 2:44:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 28, 2026 2:44:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ranger-kms/ews/webapp].
Sep 28, 2026 2:44:23 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260928 14:44:24.308831 9349 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.824s user 0.005s sys 0.242s
I20260928 14:44:24.308938 9349 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260928 14:44:24.309015 9349 mini_ranger_kms.cc:401] 127.9.33.84:35471/kms/v1/keys
Sep 28 14:44:24 dist-test-slave-pv0w krb5kdc[20092](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790606664, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.33.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 28 14:44:25 dist-test-slave-pv0w krb5kdc[20092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606664, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.33.84@KRBTEST.COM for HTTP/127.9.33.84@KRBTEST.COM
Sep 28 14:44:25 dist-test-slave-pv0w krb5kdc[20092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606664, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.33.84@KRBTEST.COM for HTTP/127.9.33.84@KRBTEST.COM
Sep 28 14:44:27 dist-test-slave-pv0w krb5kdc[20092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.9.33.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
I20260928 14:44:27.895992 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:36449
--webserver_interface=127.9.33.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:44469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:36449
--ranger_config_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.33.84:35471/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:44:27.979776 20537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:44:27.980008 20537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:44:27.980032 20537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:44:27.981355 20537 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:44:27.981402 20537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:44:27.981416 20537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:44:27.981429 20537 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:44:27.981441 20537 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:44:27.983145 20537 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:44469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/master-0/wal
--ranger_kms_url=127.9.33.84:35471/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:36449
--ranger_config_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:36449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:44:27.983433 20537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:44:27.983681 20537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:44:27.988691 20545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:44:27.988696 20542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:44:27.988696 20543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:44:27.988948 20537 server_base.cc:1034] running on GCE node
I20260928 14:44:27.989207 20537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:44:27.989516 20537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:44:27.990844 20537 hybrid_clock.cc:648] HybridClock initialized: now 1790606667990813 us; error 42 us; skew 500 ppm
Sep 28 14:44:27 dist-test-slave-pv0w krb5kdc[20092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606667, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:44:27.993706 20537 init.cc:377] Logged in from keytab as kudu/127.9.33.126@KRBTEST.COM (short username kudu)
I20260928 14:44:27.994261 20537 webserver.cc:460] Webserver started at http://127.9.33.126:38913/ using document root <none> and password file <none>
I20260928 14:44:27.994501 20537 fs_manager.cc:359] Metadata directory not provided
I20260928 14:44:27.994563 20537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:44:27.994732 20537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 14:44:27 dist-test-slave-pv0w krb5kdc[20092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606667, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for HTTP/127.9.33.84@KRBTEST.COM
I20260928 14:44:28.051831 20537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "fcf77c3247174723a615a7844e078758"
format_stamp: "Formatted at 2026-09-28 14:44:28 on dist-test-slave-pv0w"
server_key: "27ea9414e26fd093bf6bf96ced65f7b2"
server_key_iv: "71e61dc48b4a0d8ac11086c31273c005"
server_key_version: "kuduclusterkey@0"
I20260928 14:44:28.052366 20537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "fcf77c3247174723a615a7844e078758"
format_stamp: "Formatted at 2026-09-28 14:44:28 on dist-test-slave-pv0w"
server_key: "27ea9414e26fd093bf6bf96ced65f7b2"
server_key_iv: "71e61dc48b4a0d8ac11086c31273c005"
server_key_version: "kuduclusterkey@0"
I20260928 14:44:28.053961 20537 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260928 14:44:28.067173 20552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:44:28.067425 20537 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:44:28.067543 20537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "fcf77c3247174723a615a7844e078758"
format_stamp: "Formatted at 2026-09-28 14:44:28 on dist-test-slave-pv0w"
server_key: "27ea9414e26fd093bf6bf96ced65f7b2"
server_key_iv: "71e61dc48b4a0d8ac11086c31273c005"
server_key_version: "kuduclusterkey@0"
I20260928 14:44:28.067673 20537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:44:28.096268 20537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:44:28.098747 20537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:44:28.098946 20537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:44:28.104532 20537 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:36449
I20260928 14:44:28.104555 20604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:36449 every 8 connection(s)
I20260928 14:44:28.105125 20537 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:44:28.105993 20605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:44:28.109350 20605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcf77c3247174723a615a7844e078758: Bootstrap starting.
I20260928 14:44:28.110116 20605 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fcf77c3247174723a615a7844e078758: Neither blocks nor log segments found. Creating new log.
I20260928 14:44:28.110428 20605 log.cc:824] T 00000000000000000000000000000000 P fcf77c3247174723a615a7844e078758: Log is configured to *not* fsync() on all Append() calls
I20260928 14:44:28.111286 20605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcf77c3247174723a615a7844e078758: No bootstrap required, opened a new log
I20260928 14:44:28.112833 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 20537
I20260928 14:44:28.112949 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:44:28.113174 9349 external_mini_cluster.cc:1428] Setting key 0dc0be3ec845fab99541d346c74fdd98
I20260928 14:44:28.114573 20605 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcf77c3247174723a615a7844e078758 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcf77c3247174723a615a7844e078758" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 36449 } }
I20260928 14:44:28.114835 20605 raft_consensus.cc:374] T 00000000000000000000000000000000 P fcf77c3247174723a615a7844e078758 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:44:28.114889 20605 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcf77c3247174723a615a7844e078758 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcf77c3247174723a615a7844e078758, State: Initialized, Role: FOLLOWER
I20260928 14:44:28.115056 20605 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcf77c3247174723a615a7844e078758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcf77c3247174723a615a7844e078758" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 36449 } }
I20260928 14:44:28.115137 20605 raft_consensus.cc:388] T 00000000000000000000000000000000 P fcf77c3247174723a615a7844e078758 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:44:28.115166 20605 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcf77c3247174723a615a7844e078758 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:44:28.115247 20605 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcf77c3247174723a615a7844e078758 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:44:28.116026 20605 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcf77c3247174723a615a7844e078758 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcf77c3247174723a615a7844e078758" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 36449 } }
I20260928 14:44:28.116137 20605 leader_election.cc:302] T 00000000000000000000000000000000 P fcf77c3247174723a615a7844e078758 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fcf77c3247174723a615a7844e078758; no voters:
I20260928 14:44:28.116344 20605 leader_election.cc:288] T 00000000000000000000000000000000 P fcf77c3247174723a615a7844e078758 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:44:28.116406 20610 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fcf77c3247174723a615a7844e078758 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:44:28.116616 20610 raft_consensus.cc:686] T 00000000000000000000000000000000 P fcf77c3247174723a615a7844e078758 [term 1 LEADER]: Becoming Leader. State: Replica: fcf77c3247174723a615a7844e078758, State: Running, Role: LEADER
I20260928 14:44:28.116683 20605 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcf77c3247174723a615a7844e078758 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:44:28.116757 20610 consensus_queue.cc:237] T 00000000000000000000000000000000 P fcf77c3247174723a615a7844e078758 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcf77c3247174723a615a7844e078758" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 36449 } }
I20260928 14:44:28.117147 20610 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcf77c3247174723a615a7844e078758 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fcf77c3247174723a615a7844e078758" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcf77c3247174723a615a7844e078758" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 36449 } } }
I20260928 14:44:28.117292 20610 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcf77c3247174723a615a7844e078758 [sys.catalog]: This master's current role is: LEADER
I20260928 14:44:28.117467 20610 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcf77c3247174723a615a7844e078758 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fcf77c3247174723a615a7844e078758. Latest consensus state: current_term: 1 leader_uuid: "fcf77c3247174723a615a7844e078758" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcf77c3247174723a615a7844e078758" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 36449 } } }
I20260928 14:44:28.117570 20610 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcf77c3247174723a615a7844e078758 [sys.catalog]: This master's current role is: LEADER
I20260928 14:44:28.117713 20619 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:44:28.118273 20619 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:44:28.118566 20605 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Sep 28 14:44:28 dist-test-slave-pv0w krb5kdc[20092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:44:28.125028 20627 mvcc.cc:205] Tried to move back new op lower bound from 7334324912633364480 to 7334324912606760960. Current Snapshot: MvccSnapshot[applied={T|T < 7334324912633364480}]
I20260928 14:44:28.126235 20619 catalog_manager.cc:1347] Generated new cluster ID: 9d3d1b582e284b3a87d26231d33e45b4
I20260928 14:44:28.126411 20619 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:44:28.127816 20609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:44:28.113711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35734 (local address 127.9.33.126:36449)
0928 14:44:28.113895 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0928 14:44:28.113906 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:44:28.113945 (+ 39us) server_negotiation.cc:407] Connection header received
0928 14:44:28.114545 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:44:28.114552 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:44:28.114614 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:44:28.114764 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:44:28.115560 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:44:28.116916 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:44:28.118035 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:44:28.118639 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:44:28.121491 (+ 2852us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:44:28.121965 (+ 474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:44:28.121969 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:44:28.121969 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:44:28.121981 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:44:28.125142 (+ 3161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:44:28.125651 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:44:28.125654 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:44:28.125654 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:44:28.125809 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:44:28.126053 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:44:28.126054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:44:28.126054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:44:28.126519 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:44:28.127125 (+ 606us) server_negotiation.cc:1035] Waiting for connection context
0928 14:44:28.127195 (+ 70us) server_negotiation.cc:299] Negotiation successful
0928 14:44:28.127496 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":78,"threads_started":1}
I20260928 14:44:28.138458 20619 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:44:28.139338 20619 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:44:28.150511 20619 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fcf77c3247174723a615a7844e078758: Generated new TSK 0
I20260928 14:44:28.150828 20619 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.33.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab.
Entry for principal kudu/127.9.33.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab.
I20260928 14:44:28.888075 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.33.84:35471/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.65:0
--local_ip_for_outbound_sockets=127.9.33.65
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:36449
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:44469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:44:29.016348 20650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:44:29.016594 20650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:44:29.016639 20650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:44:29.018105 20650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:44:29.018172 20650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:44:29.018220 20650 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.65
I20260928 14:44:29.020272 20650 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:44469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.9.33.84:35471/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:36449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.65
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:44:29.020582 20650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:44:29.020867 20650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:44:29.028275 20655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:44:29.028250 20658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:44:29.028322 20656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:44:29.028635 20650 server_base.cc:1034] running on GCE node
I20260928 14:44:29.028842 20650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:44:29.029148 20650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:44:29.030331 20650 hybrid_clock.cc:648] HybridClock initialized: now 1790606669030305 us; error 45 us; skew 500 ppm
Sep 28 14:44:29 dist-test-slave-pv0w krb5kdc[20092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606669, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:44:29.042323 20650 init.cc:377] Logged in from keytab as kudu/127.9.33.65@KRBTEST.COM (short username kudu)
I20260928 14:44:29.042979 20650 webserver.cc:460] Webserver started at http://127.9.33.65:34235/ using document root <none> and password file <none>
I20260928 14:44:29.043332 20650 fs_manager.cc:359] Metadata directory not provided
I20260928 14:44:29.043419 20650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:44:29.043633 20650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 14:44:29 dist-test-slave-pv0w krb5kdc[20092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606669, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for HTTP/127.9.33.84@KRBTEST.COM
I20260928 14:44:29.060280 20650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-0/data/instance:
uuid: "1c125a802e6a447cb147f24e2f2f7b2d"
format_stamp: "Formatted at 2026-09-28 14:44:29 on dist-test-slave-pv0w"
server_key: "cbaac9b358fe2e2e9d870b0de8540aa3"
server_key_iv: "ebe941681c625075191519c96d56cff3"
server_key_version: "kuduclusterkey@0"
I20260928 14:44:29.060828 20650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance:
uuid: "1c125a802e6a447cb147f24e2f2f7b2d"
format_stamp: "Formatted at 2026-09-28 14:44:29 on dist-test-slave-pv0w"
server_key: "cbaac9b358fe2e2e9d870b0de8540aa3"
server_key_iv: "ebe941681c625075191519c96d56cff3"
server_key_version: "kuduclusterkey@0"
I20260928 14:44:29.062995 20650 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260928 14:44:29.081730 20667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:44:29.081985 20650 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:44:29.082093 20650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-0/wal
uuid: "1c125a802e6a447cb147f24e2f2f7b2d"
format_stamp: "Formatted at 2026-09-28 14:44:29 on dist-test-slave-pv0w"
server_key: "cbaac9b358fe2e2e9d870b0de8540aa3"
server_key_iv: "ebe941681c625075191519c96d56cff3"
server_key_version: "kuduclusterkey@0"
I20260928 14:44:29.082198 20650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:44:29.108233 20650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:44:29.110903 20650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:44:29.111109 20650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:44:29.111356 20650 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:44:29.111785 20650 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:44:29.111877 20650 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:44:29.111927 20650 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:44:29.111961 20650 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:44:29.118932 20650 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.65:43827
I20260928 14:44:29.119447 20650 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
I20260928 14:44:29.125936 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 20650
I20260928 14:44:29.126081 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance
I20260928 14:44:29.126055 20780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.65:43827 every 8 connection(s)
I20260928 14:44:29.126319 9349 external_mini_cluster.cc:1428] Setting key e180e39972d40404b7ad2127c27e2089
Sep 28 14:44:29 dist-test-slave-pv0w krb5kdc[20092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606669, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.33.66@KRBTEST.COM; defaulting to no policy
I20260928 14:44:29.160673 20785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:44:29.137122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:34227 (local address 127.9.33.126:36449)
0928 14:44:29.146095 (+ 8973us) server_negotiation.cc:206] Beginning negotiation
0928 14:44:29.146098 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:44:29.146107 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:44:29.146136 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:44:29.146137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:44:29.146151 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:44:29.146218 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:44:29.147102 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:44:29.147901 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:44:29.149391 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:44:29.149711 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:44:29.149780 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:44:29.156689 (+ 6909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:44:29.156692 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:44:29.156692 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:44:29.156701 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:44:29.158719 (+ 2018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:44:29.159280 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:44:29.159282 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:44:29.159282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:44:29.159350 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:44:29.159694 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:44:29.159696 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:44:29.159696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:44:29.159910 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:44:29.159994 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0928 14:44:29.160449 (+ 455us) server_negotiation.cc:299] Negotiation successful
0928 14:44:29.160564 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8936,"thread_start_us":84,"threads_started":1}
I20260928 14:44:29.161055 20784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:44:29.136732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:36449 (local address 127.9.33.65:34227)
0928 14:44:29.145099 (+ 8367us) negotiation.cc:107] Waiting for socket to connect
0928 14:44:29.145177 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0928 14:44:29.145856 (+ 679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:44:29.146371 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:44:29.146378 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:44:29.146658 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:44:29.146965 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:44:29.146972 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:44:29.148019 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:44:29.148020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:44:29.149288 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:44:29.149291 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:44:29.149860 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:44:29.149861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:44:29.150052 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:44:29.154606 (+ 4554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:44:29.154620 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:44:29.156538 (+ 1918us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:44:29.158836 (+ 2298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:44:29.158847 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:44:29.158850 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:44:29.159184 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:44:29.159424 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:44:29.159425 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:44:29.159425 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:44:29.159596 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:44:29.160004 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:44:29.160012 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:44:29.160258 (+ 246us) client_negotiation.cc:769] Sending connection context
0928 14:44:29.160403 (+ 145us) client_negotiation.cc:240] Negotiation successful
0928 14:44:29.160617 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8294,"thread_start_us":122,"threads_started":1}
I20260928 14:44:29.161628 20781 heartbeater.cc:347] Connected to a master server at 127.9.33.126:36449
I20260928 14:44:29.161747 20781 heartbeater.cc:464] Registering TS with master...
I20260928 14:44:29.161990 20781 heartbeater.cc:511] Master 127.9.33.126:36449 requested a full tablet report, sending...
I20260928 14:44:29.162649 20569 ts_manager.cc:194] Registered new tserver with Master: 1c125a802e6a447cb147f24e2f2f7b2d (127.9.33.65:43827)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.66@KRBTEST.COM" created.
I20260928 14:44:29.163499 20569 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.65@KRBTEST.COM'} at 127.9.33.65:34227
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab.
Entry for principal kudu/127.9.33.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab.
I20260928 14:44:29.190457 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.33.84:35471/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.66:0
--local_ip_for_outbound_sockets=127.9.33.66
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:36449
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:44469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 14:44:29 dist-test-slave-pv0w krb5kdc[20092](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790606669, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260928 14:44:29.297530 20789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:44:29.297847 20789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:44:29.297883 20789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:44:29.299690 20789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:44:29.299787 20789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:44:29.299830 20789 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.66
I20260928 14:44:29.302610 20789 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:44469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.9.33.84:35471/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:36449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.66
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:44:29.302964 20789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:44:29.303272 20789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:44:29.309238 20794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:44:29.309238 20795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:44:29.309494 20789 server_base.cc:1034] running on GCE node
W20260928 14:44:29.309544 20797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:44:29.309785 20789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:44:29.310021 20789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:44:29.311259 20789 hybrid_clock.cc:648] HybridClock initialized: now 1790606669311240 us; error 37 us; skew 500 ppm
Sep 28 14:44:29 dist-test-slave-pv0w krb5kdc[20092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606669, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:44:29.314193 20789 init.cc:377] Logged in from keytab as kudu/127.9.33.66@KRBTEST.COM (short username kudu)
I20260928 14:44:29.314778 20789 webserver.cc:460] Webserver started at http://127.9.33.66:44393/ using document root <none> and password file <none>
I20260928 14:44:29.315063 20789 fs_manager.cc:359] Metadata directory not provided
I20260928 14:44:29.315135 20789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:44:29.315299 20789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 14:44:29 dist-test-slave-pv0w krb5kdc[20092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606669, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for HTTP/127.9.33.84@KRBTEST.COM
I20260928 14:44:29.329675 20789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-1/data/instance:
uuid: "7c54f609458642ca9897be82622f195b"
format_stamp: "Formatted at 2026-09-28 14:44:29 on dist-test-slave-pv0w"
server_key: "2a296747fa35e3ef49661740f57c5fd9"
server_key_iv: "7ccbfe479003defc870e813bb04270c2"
server_key_version: "kuduclusterkey@0"
I20260928 14:44:29.330186 20789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance:
uuid: "7c54f609458642ca9897be82622f195b"
format_stamp: "Formatted at 2026-09-28 14:44:29 on dist-test-slave-pv0w"
server_key: "2a296747fa35e3ef49661740f57c5fd9"
server_key_iv: "7ccbfe479003defc870e813bb04270c2"
server_key_version: "kuduclusterkey@0"
I20260928 14:44:29.332094 20789 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260928 14:44:29.347234 20805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:44:29.347487 20789 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:44:29.347574 20789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-1/wal
uuid: "7c54f609458642ca9897be82622f195b"
format_stamp: "Formatted at 2026-09-28 14:44:29 on dist-test-slave-pv0w"
server_key: "2a296747fa35e3ef49661740f57c5fd9"
server_key_iv: "7ccbfe479003defc870e813bb04270c2"
server_key_version: "kuduclusterkey@0"
I20260928 14:44:29.347668 20789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:44:29.374648 20789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:44:29.377343 20789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:44:29.377563 20789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:44:29.377830 20789 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:44:29.378255 20789 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:44:29.378342 20789 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:44:29.378415 20789 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:44:29.378475 20789 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:44:29.393656 20789 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.66:43975
I20260928 14:44:29.394233 20789 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
I20260928 14:44:29.394809 20918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.66:43975 every 8 connection(s)
I20260928 14:44:29.399897 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 20789
I20260928 14:44:29.400029 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance
I20260928 14:44:29.400229 9349 external_mini_cluster.cc:1428] Setting key 00034d6dd01fc9c5634c3d6adf5675f3
Sep 28 14:44:29 dist-test-slave-pv0w krb5kdc[20092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606669, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.33.67@KRBTEST.COM; defaulting to no policy
I20260928 14:44:29.411425 20785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:44:29.395864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:34931 (local address 127.9.33.126:36449)
0928 14:44:29.395929 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0928 14:44:29.395932 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:44:29.400083 (+ 4151us) server_negotiation.cc:407] Connection header received
0928 14:44:29.400959 (+ 876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:44:29.400960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:44:29.400972 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:44:29.401019 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:44:29.401834 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:44:29.402686 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:44:29.404327 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:44:29.404697 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:44:29.404744 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:44:29.407802 (+ 3058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:44:29.407804 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:44:29.407805 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:44:29.407814 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:44:29.409715 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:44:29.410270 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:44:29.410272 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:44:29.410273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:44:29.410337 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:44:29.410670 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:44:29.410671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:44:29.410671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:44:29.410826 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:44:29.410886 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0928 14:44:29.411233 (+ 347us) server_negotiation.cc:299] Negotiation successful
0928 14:44:29.411330 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260928 14:44:29.411639 20921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:44:29.395540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:36449 (local address 127.9.33.66:34931)
0928 14:44:29.399885 (+ 4345us) negotiation.cc:107] Waiting for socket to connect
0928 14:44:29.399952 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0928 14:44:29.400816 (+ 864us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:44:29.401101 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:44:29.401106 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:44:29.401379 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:44:29.401709 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:44:29.401717 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:44:29.402808 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:44:29.402809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:44:29.404220 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:44:29.404224 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:44:29.404844 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:44:29.404845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:44:29.405015 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:44:29.405708 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:44:29.405720 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:44:29.407647 (+ 1927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:44:29.409833 (+ 2186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:44:29.409842 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:44:29.409845 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:44:29.410172 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:44:29.410412 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:44:29.410413 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:44:29.410413 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:44:29.410584 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:44:29.410916 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:44:29.410922 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:44:29.411074 (+ 152us) client_negotiation.cc:769] Sending connection context
0928 14:44:29.411194 (+ 120us) client_negotiation.cc:240] Negotiation successful
0928 14:44:29.411360 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4285,"thread_start_us":97,"threads_started":1}
I20260928 14:44:29.412393 20919 heartbeater.cc:347] Connected to a master server at 127.9.33.126:36449
I20260928 14:44:29.412539 20919 heartbeater.cc:464] Registering TS with master...
I20260928 14:44:29.412737 20919 heartbeater.cc:511] Master 127.9.33.126:36449 requested a full tablet report, sending...
I20260928 14:44:29.413169 20569 ts_manager.cc:194] Registered new tserver with Master: 7c54f609458642ca9897be82622f195b (127.9.33.66:43975)
I20260928 14:44:29.413677 20569 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.66@KRBTEST.COM'} at 127.9.33.66:34931
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab.
Entry for principal kudu/127.9.33.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab.
I20260928 14:44:29.431062 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.33.84:35471/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.67:0
--local_ip_for_outbound_sockets=127.9.33.67
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:36449
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:44469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:44:29.530287 20924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:44:29.530532 20924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:44:29.530573 20924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:44:29.531939 20924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:44:29.532011 20924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:44:29.532058 20924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.67
I20260928 14:44:29.534415 20924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:44469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.9.33.84:35471/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:36449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.67
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:44:29.534696 20924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:44:29.535070 20924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:44:29.541855 20932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:44:29.542088 20924 server_base.cc:1034] running on GCE node
W20260928 14:44:29.541870 20930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:44:29.542186 20929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:44:29.542541 20924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:44:29.542848 20924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:44:29.544023 20924 hybrid_clock.cc:648] HybridClock initialized: now 1790606669543979 us; error 46 us; skew 500 ppm
Sep 28 14:44:29 dist-test-slave-pv0w krb5kdc[20092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606669, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:44:29.547539 20924 init.cc:377] Logged in from keytab as kudu/127.9.33.67@KRBTEST.COM (short username kudu)
I20260928 14:44:29.548074 20924 webserver.cc:460] Webserver started at http://127.9.33.67:44441/ using document root <none> and password file <none>
I20260928 14:44:29.548331 20924 fs_manager.cc:359] Metadata directory not provided
I20260928 14:44:29.548399 20924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:44:29.548558 20924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 14:44:29 dist-test-slave-pv0w krb5kdc[20092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606669, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for HTTP/127.9.33.84@KRBTEST.COM
I20260928 14:44:29.566008 20924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-2/data/instance:
uuid: "a7f4643e91ff415db4c0f5db4e9e0e87"
format_stamp: "Formatted at 2026-09-28 14:44:29 on dist-test-slave-pv0w"
server_key: "46457055d48e42fba26f377ae828d0b4"
server_key_iv: "0cb4aa5e648d1798d8f3cc2894ce4546"
server_key_version: "kuduclusterkey@0"
I20260928 14:44:29.567008 20924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance:
uuid: "a7f4643e91ff415db4c0f5db4e9e0e87"
format_stamp: "Formatted at 2026-09-28 14:44:29 on dist-test-slave-pv0w"
server_key: "46457055d48e42fba26f377ae828d0b4"
server_key_iv: "0cb4aa5e648d1798d8f3cc2894ce4546"
server_key_version: "kuduclusterkey@0"
I20260928 14:44:29.576301 20924 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.002s sys 0.000s
I20260928 14:44:29.601468 20939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:44:29.601737 20924 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260928 14:44:29.601837 20924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-2/wal
uuid: "a7f4643e91ff415db4c0f5db4e9e0e87"
format_stamp: "Formatted at 2026-09-28 14:44:29 on dist-test-slave-pv0w"
server_key: "46457055d48e42fba26f377ae828d0b4"
server_key_iv: "0cb4aa5e648d1798d8f3cc2894ce4546"
server_key_version: "kuduclusterkey@0"
I20260928 14:44:29.601936 20924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:44:29.617551 20924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:44:29.620144 20924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:44:29.620376 20924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:44:29.620677 20924 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:44:29.621099 20924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:44:29.621189 20924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:44:29.621263 20924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:44:29.621320 20924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:44:29.629995 20924 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.67:39301
I20260928 14:44:29.630434 21052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.67:39301 every 8 connection(s)
I20260928 14:44:29.630523 20924 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
I20260928 14:44:29.634954 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 20924
I20260928 14:44:29.635072 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance
I20260928 14:44:29.635272 9349 external_mini_cluster.cc:1428] Setting key 6c6f5a7ffea468d188451d50c202fa9e
Sep 28 14:44:29 dist-test-slave-pv0w krb5kdc[20092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606669, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:44:29.644526 20785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:44:29.631653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:42545 (local address 127.9.33.126:36449)
0928 14:44:29.631696 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 14:44:29.631698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:44:29.631709 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:44:29.632428 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:44:29.632430 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:44:29.632443 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:44:29.632507 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:44:29.633256 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:44:29.634077 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:44:29.637430 (+ 3353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:44:29.637740 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:44:29.637792 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:44:29.640701 (+ 2909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:44:29.640703 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:44:29.640703 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:44:29.640712 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:44:29.642603 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:44:29.643188 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:44:29.643190 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:44:29.643191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:44:29.643278 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:44:29.643628 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:44:29.643629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:44:29.643630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:44:29.643812 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:44:29.643871 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0928 14:44:29.644326 (+ 455us) server_negotiation.cc:299] Negotiation successful
0928 14:44:29.644423 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 14:44:29.644694 21055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:44:29.631358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:36449 (local address 127.9.33.67:42545)
0928 14:44:29.631554 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0928 14:44:29.631606 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0928 14:44:29.632291 (+ 685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:44:29.632600 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:44:29.632604 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:44:29.632858 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:44:29.633153 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:44:29.633159 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:44:29.634192 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:44:29.634193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:44:29.637309 (+ 3116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:44:29.637313 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:44:29.637896 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:44:29.637897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:44:29.638082 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:44:29.638777 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:44:29.638789 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:44:29.640570 (+ 1781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:44:29.642711 (+ 2141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:44:29.642721 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:44:29.642724 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:44:29.643072 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:44:29.643366 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:44:29.643367 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:44:29.643368 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:44:29.643537 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:44:29.643908 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:44:29.643915 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:44:29.644133 (+ 218us) client_negotiation.cc:769] Sending connection context
0928 14:44:29.644256 (+ 123us) client_negotiation.cc:240] Negotiation successful
0928 14:44:29.644440 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":86,"threads_started":1}
I20260928 14:44:29.645210 21053 heartbeater.cc:347] Connected to a master server at 127.9.33.126:36449
I20260928 14:44:29.645327 21053 heartbeater.cc:464] Registering TS with master...
I20260928 14:44:29.645531 21053 heartbeater.cc:511] Master 127.9.33.126:36449 requested a full tablet report, sending...
I20260928 14:44:29.646003 20569 ts_manager.cc:194] Registered new tserver with Master: a7f4643e91ff415db4c0f5db4e9e0e87 (127.9.33.67:39301)
I20260928 14:44:29.646551 20569 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.67@KRBTEST.COM'} at 127.9.33.67:42545
I20260928 14:44:29.647408 9349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:44:29.657362 20785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:44:29.649249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35736 (local address 127.9.33.126:36449)
0928 14:44:29.649298 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0928 14:44:29.649300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:44:29.649428 (+ 128us) server_negotiation.cc:407] Connection header received
0928 14:44:29.649532 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:44:29.649533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:44:29.649557 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:44:29.649589 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:44:29.650400 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:44:29.651135 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:44:29.652259 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:44:29.652558 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:44:29.652605 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:44:29.654260 (+ 1655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:44:29.654262 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:44:29.654262 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:44:29.654270 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:44:29.656077 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:44:29.656489 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:44:29.656490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:44:29.656491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:44:29.656563 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:44:29.656775 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:44:29.656776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:44:29.656776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:44:29.656901 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:44:29.656973 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 14:44:29.657191 (+ 218us) server_negotiation.cc:299] Negotiation successful
0928 14:44:29.657269 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260928 14:44:29.659507 20569 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35736:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:44:29.661968 20569 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:44:29.677089 21066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:44:29.664901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:39301 (local address 127.0.0.1:44678)
0928 14:44:29.668082 (+ 3181us) negotiation.cc:107] Waiting for socket to connect
0928 14:44:29.668134 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0928 14:44:29.668227 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:44:29.668450 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:44:29.668454 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:44:29.668456 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:44:29.668557 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:44:29.668560 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:44:29.673425 (+ 4865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:44:29.673426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:44:29.675567 (+ 2141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:44:29.675571 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:44:29.676767 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:44:29.676768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:44:29.676854 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:44:29.676869 (+ 15us) client_negotiation.cc:769] Sending connection context
0928 14:44:29.676940 (+ 71us) client_negotiation.cc:240] Negotiation successful
0928 14:44:29.676995 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3144,"thread_start_us":105,"threads_started":1}
I20260928 14:44:29.677629 21067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:44:29.665174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:43827 (local address 127.0.0.1:54998)
0928 14:44:29.672000 (+ 6826us) negotiation.cc:107] Waiting for socket to connect
0928 14:44:29.672009 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 14:44:29.672063 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:44:29.672324 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:44:29.672325 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:44:29.672327 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:44:29.672366 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:44:29.672367 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:44:29.673574 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:44:29.673575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:44:29.675448 (+ 1873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:44:29.675451 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:44:29.676951 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:44:29.676952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:44:29.677010 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:44:29.677015 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:44:29.677065 (+ 50us) client_negotiation.cc:240] Negotiation successful
0928 14:44:29.677105 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6812,"mutex_wait_us":608,"thread_start_us":69,"threads_started":1}
I20260928 14:44:29.678066 21068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:44:29.665488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44678 (local address 127.9.33.67:39301)
0928 14:44:29.666170 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0928 14:44:29.666176 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:44:29.668232 (+ 2056us) server_negotiation.cc:407] Connection header received
0928 14:44:29.668341 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:44:29.668344 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:44:29.668363 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:44:29.668444 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:44:29.668725 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:44:29.669912 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:44:29.675736 (+ 5824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:44:29.676560 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:44:29.676639 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:44:29.676730 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0928 14:44:29.677934 (+ 1204us) server_negotiation.cc:299] Negotiation successful
0928 14:44:29.677983 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":84,"threads_started":1}
I20260928 14:44:29.678893 20987 tablet_service.cc:1467] Processing CreateTablet for tablet 837f7511dca143ba8e45d34a2d2f1d60 (DEFAULT_TABLE table=test-table [id=c104fab11b35491cadfc2632c297d268]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:44:29.679299 20987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 837f7511dca143ba8e45d34a2d2f1d60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:44:29.679399 21070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:44:29.667883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54998 (local address 127.9.33.65:43827)
0928 14:44:29.668375 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0928 14:44:29.668380 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:44:29.672125 (+ 3745us) server_negotiation.cc:407] Connection header received
0928 14:44:29.672165 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:44:29.672168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:44:29.672185 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:44:29.672254 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:44:29.672436 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:44:29.673486 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:44:29.675570 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:44:29.676795 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:44:29.676861 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:44:29.676917 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 14:44:29.679269 (+ 2352us) server_negotiation.cc:299] Negotiation successful
0928 14:44:29.679321 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":65,"threads_started":1}
I20260928 14:44:29.680042 20710 tablet_service.cc:1467] Processing CreateTablet for tablet 837f7511dca143ba8e45d34a2d2f1d60 (DEFAULT_TABLE table=test-table [id=c104fab11b35491cadfc2632c297d268]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:44:29.680343 20710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 837f7511dca143ba8e45d34a2d2f1d60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:44:29.684094 21065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:44:29.664906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:43975 (local address 127.0.0.1:43102)
0928 14:44:29.671810 (+ 6904us) negotiation.cc:107] Waiting for socket to connect
0928 14:44:29.671829 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 14:44:29.671908 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:44:29.672180 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:44:29.672181 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:44:29.672183 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:44:29.672239 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:44:29.672241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:44:29.673786 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:44:29.673786 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:44:29.678149 (+ 4363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:44:29.678152 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:44:29.683808 (+ 5656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:44:29.683809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:44:29.683897 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:44:29.683903 (+ 6us) client_negotiation.cc:769] Sending connection context
0928 14:44:29.683955 (+ 52us) client_negotiation.cc:240] Negotiation successful
0928 14:44:29.684006 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6884,"mutex_wait_us":3232,"thread_start_us":97,"threads_started":1}
I20260928 14:44:29.684413 21069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:44:29.665712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43102 (local address 127.9.33.66:43975)
0928 14:44:29.666250 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0928 14:44:29.666254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:44:29.671907 (+ 5653us) server_negotiation.cc:407] Connection header received
0928 14:44:29.671984 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:44:29.671987 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:44:29.672005 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:44:29.672089 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:44:29.672314 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:44:29.673704 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:44:29.678283 (+ 4579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:44:29.679077 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:44:29.679149 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:44:29.679223 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 14:44:29.684274 (+ 5051us) server_negotiation.cc:299] Negotiation successful
0928 14:44:29.684331 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":75,"threads_started":1}
I20260928 14:44:29.684908 21071 tablet_bootstrap.cc:492] T 837f7511dca143ba8e45d34a2d2f1d60 P a7f4643e91ff415db4c0f5db4e9e0e87: Bootstrap starting.
I20260928 14:44:29.684993 20843 tablet_service.cc:1467] Processing CreateTablet for tablet 837f7511dca143ba8e45d34a2d2f1d60 (DEFAULT_TABLE table=test-table [id=c104fab11b35491cadfc2632c297d268]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:44:29.685319 20843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 837f7511dca143ba8e45d34a2d2f1d60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:44:29.686272 21071 tablet_bootstrap.cc:654] T 837f7511dca143ba8e45d34a2d2f1d60 P a7f4643e91ff415db4c0f5db4e9e0e87: Neither blocks nor log segments found. Creating new log.
I20260928 14:44:29.686713 21071 log.cc:824] T 837f7511dca143ba8e45d34a2d2f1d60 P a7f4643e91ff415db4c0f5db4e9e0e87: Log is configured to *not* fsync() on all Append() calls
I20260928 14:44:29.687573 21071 tablet_bootstrap.cc:492] T 837f7511dca143ba8e45d34a2d2f1d60 P a7f4643e91ff415db4c0f5db4e9e0e87: No bootstrap required, opened a new log
I20260928 14:44:29.687640 21071 ts_tablet_manager.cc:1397] T 837f7511dca143ba8e45d34a2d2f1d60 P a7f4643e91ff415db4c0f5db4e9e0e87: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260928 14:44:29.689143 21072 tablet_bootstrap.cc:492] T 837f7511dca143ba8e45d34a2d2f1d60 P 1c125a802e6a447cb147f24e2f2f7b2d: Bootstrap starting.
I20260928 14:44:29.689600 21071 raft_consensus.cc:348] T 837f7511dca143ba8e45d34a2d2f1d60 P a7f4643e91ff415db4c0f5db4e9e0e87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7f4643e91ff415db4c0f5db4e9e0e87" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39301 } } peers { permanent_uuid: "7c54f609458642ca9897be82622f195b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 43975 } } peers { permanent_uuid: "1c125a802e6a447cb147f24e2f2f7b2d" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 43827 } }
I20260928 14:44:29.689766 21071 raft_consensus.cc:374] T 837f7511dca143ba8e45d34a2d2f1d60 P a7f4643e91ff415db4c0f5db4e9e0e87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:44:29.689798 21071 raft_consensus.cc:729] T 837f7511dca143ba8e45d34a2d2f1d60 P a7f4643e91ff415db4c0f5db4e9e0e87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7f4643e91ff415db4c0f5db4e9e0e87, State: Initialized, Role: FOLLOWER
I20260928 14:44:29.689911 21071 consensus_queue.cc:260] T 837f7511dca143ba8e45d34a2d2f1d60 P a7f4643e91ff415db4c0f5db4e9e0e87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7f4643e91ff415db4c0f5db4e9e0e87" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39301 } } peers { permanent_uuid: "7c54f609458642ca9897be82622f195b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 43975 } } peers { permanent_uuid: "1c125a802e6a447cb147f24e2f2f7b2d" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 43827 } }
I20260928 14:44:29.689945 21072 tablet_bootstrap.cc:654] T 837f7511dca143ba8e45d34a2d2f1d60 P 1c125a802e6a447cb147f24e2f2f7b2d: Neither blocks nor log segments found. Creating new log.
I20260928 14:44:29.690160 21071 ts_tablet_manager.cc:1428] T 837f7511dca143ba8e45d34a2d2f1d60 P a7f4643e91ff415db4c0f5db4e9e0e87: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260928 14:44:29.690223 21072 log.cc:824] T 837f7511dca143ba8e45d34a2d2f1d60 P 1c125a802e6a447cb147f24e2f2f7b2d: Log is configured to *not* fsync() on all Append() calls
I20260928 14:44:29.690548 21074 tablet_bootstrap.cc:492] T 837f7511dca143ba8e45d34a2d2f1d60 P 7c54f609458642ca9897be82622f195b: Bootstrap starting.
I20260928 14:44:29.690739 21053 heartbeater.cc:503] Master 127.9.33.126:36449 was elected leader, sending a full tablet report...
I20260928 14:44:29.691506 21074 tablet_bootstrap.cc:654] T 837f7511dca143ba8e45d34a2d2f1d60 P 7c54f609458642ca9897be82622f195b: Neither blocks nor log segments found. Creating new log.
I20260928 14:44:29.691941 21074 log.cc:824] T 837f7511dca143ba8e45d34a2d2f1d60 P 7c54f609458642ca9897be82622f195b: Log is configured to *not* fsync() on all Append() calls
I20260928 14:44:29.692739 21072 tablet_bootstrap.cc:492] T 837f7511dca143ba8e45d34a2d2f1d60 P 1c125a802e6a447cb147f24e2f2f7b2d: No bootstrap required, opened a new log
I20260928 14:44:29.692812 21072 ts_tablet_manager.cc:1397] T 837f7511dca143ba8e45d34a2d2f1d60 P 1c125a802e6a447cb147f24e2f2f7b2d: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260928 14:44:29.695845 21072 raft_consensus.cc:348] T 837f7511dca143ba8e45d34a2d2f1d60 P 1c125a802e6a447cb147f24e2f2f7b2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7f4643e91ff415db4c0f5db4e9e0e87" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39301 } } peers { permanent_uuid: "7c54f609458642ca9897be82622f195b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 43975 } } peers { permanent_uuid: "1c125a802e6a447cb147f24e2f2f7b2d" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 43827 } }
I20260928 14:44:29.696043 21072 raft_consensus.cc:374] T 837f7511dca143ba8e45d34a2d2f1d60 P 1c125a802e6a447cb147f24e2f2f7b2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:44:29.696082 21072 raft_consensus.cc:729] T 837f7511dca143ba8e45d34a2d2f1d60 P 1c125a802e6a447cb147f24e2f2f7b2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c125a802e6a447cb147f24e2f2f7b2d, State: Initialized, Role: FOLLOWER
I20260928 14:44:29.696224 21072 consensus_queue.cc:260] T 837f7511dca143ba8e45d34a2d2f1d60 P 1c125a802e6a447cb147f24e2f2f7b2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7f4643e91ff415db4c0f5db4e9e0e87" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39301 } } peers { permanent_uuid: "7c54f609458642ca9897be82622f195b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 43975 } } peers { permanent_uuid: "1c125a802e6a447cb147f24e2f2f7b2d" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 43827 } }
I20260928 14:44:29.696502 21072 ts_tablet_manager.cc:1428] T 837f7511dca143ba8e45d34a2d2f1d60 P 1c125a802e6a447cb147f24e2f2f7b2d: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260928 14:44:29.696549 20781 heartbeater.cc:503] Master 127.9.33.126:36449 was elected leader, sending a full tablet report...
I20260928 14:44:29.696970 21074 tablet_bootstrap.cc:492] T 837f7511dca143ba8e45d34a2d2f1d60 P 7c54f609458642ca9897be82622f195b: No bootstrap required, opened a new log
I20260928 14:44:29.697045 21074 ts_tablet_manager.cc:1397] T 837f7511dca143ba8e45d34a2d2f1d60 P 7c54f609458642ca9897be82622f195b: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20260928 14:44:29.699057 21074 raft_consensus.cc:348] T 837f7511dca143ba8e45d34a2d2f1d60 P 7c54f609458642ca9897be82622f195b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7f4643e91ff415db4c0f5db4e9e0e87" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39301 } } peers { permanent_uuid: "7c54f609458642ca9897be82622f195b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 43975 } } peers { permanent_uuid: "1c125a802e6a447cb147f24e2f2f7b2d" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 43827 } }
I20260928 14:44:29.699234 21074 raft_consensus.cc:374] T 837f7511dca143ba8e45d34a2d2f1d60 P 7c54f609458642ca9897be82622f195b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:44:29.699263 21074 raft_consensus.cc:729] T 837f7511dca143ba8e45d34a2d2f1d60 P 7c54f609458642ca9897be82622f195b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c54f609458642ca9897be82622f195b, State: Initialized, Role: FOLLOWER
I20260928 14:44:29.699404 21074 consensus_queue.cc:260] T 837f7511dca143ba8e45d34a2d2f1d60 P 7c54f609458642ca9897be82622f195b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7f4643e91ff415db4c0f5db4e9e0e87" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39301 } } peers { permanent_uuid: "7c54f609458642ca9897be82622f195b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 43975 } } peers { permanent_uuid: "1c125a802e6a447cb147f24e2f2f7b2d" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 43827 } }
I20260928 14:44:29.699739 20919 heartbeater.cc:503] Master 127.9.33.126:36449 was elected leader, sending a full tablet report...
I20260928 14:44:29.699679 21074 ts_tablet_manager.cc:1428] T 837f7511dca143ba8e45d34a2d2f1d60 P 7c54f609458642ca9897be82622f195b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
Sep 28 14:44:29 dist-test-slave-pv0w krb5kdc[20092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606669, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for HTTP/127.9.33.84@KRBTEST.COM
W20260928 14:44:29.876847 20782 tablet.cc:2369] T 837f7511dca143ba8e45d34a2d2f1d60 P 1c125a802e6a447cb147f24e2f2f7b2d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:44:29.881398 21054 tablet.cc:2369] T 837f7511dca143ba8e45d34a2d2f1d60 P a7f4643e91ff415db4c0f5db4e9e0e87: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:44:29.896955 20920 tablet.cc:2369] T 837f7511dca143ba8e45d34a2d2f1d60 P 7c54f609458642ca9897be82622f195b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:44:29.937160 21079 raft_consensus.cc:482] T 837f7511dca143ba8e45d34a2d2f1d60 P 7c54f609458642ca9897be82622f195b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:44:29.937310 21079 raft_consensus.cc:504] T 837f7511dca143ba8e45d34a2d2f1d60 P 7c54f609458642ca9897be82622f195b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7f4643e91ff415db4c0f5db4e9e0e87" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39301 } } peers { permanent_uuid: "7c54f609458642ca9897be82622f195b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 43975 } } peers { permanent_uuid: "1c125a802e6a447cb147f24e2f2f7b2d" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 43827 } }
I20260928 14:44:29.937692 21079 leader_election.cc:288] T 837f7511dca143ba8e45d34a2d2f1d60 P 7c54f609458642ca9897be82622f195b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a7f4643e91ff415db4c0f5db4e9e0e87 (127.9.33.67:39301), 1c125a802e6a447cb147f24e2f2f7b2d (127.9.33.65:43827)
I20260928 14:44:29.942488 21068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:44:29.938253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:43891 (local address 127.9.33.67:39301)
0928 14:44:29.938301 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0928 14:44:29.938303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:44:29.938643 (+ 340us) server_negotiation.cc:407] Connection header received
0928 14:44:29.938751 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:44:29.938752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:44:29.938761 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:44:29.938790 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:44:29.939000 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:44:29.939839 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:44:29.941486 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:44:29.942044 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:44:29.942114 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:44:29.942126 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0928 14:44:29.942373 (+ 247us) server_negotiation.cc:299] Negotiation successful
0928 14:44:29.942416 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 14:44:29.942940 21081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:44:29.937929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:39301 (local address 127.9.33.66:43891)
0928 14:44:29.938572 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0928 14:44:29.938584 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:44:29.938674 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:44:29.938858 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:44:29.938859 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:44:29.938861 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:44:29.938930 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:44:29.938932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:44:29.939958 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:44:29.939959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:44:29.941229 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:44:29.941231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:44:29.942218 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:44:29.942219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:44:29.942292 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:44:29.942295 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:44:29.942811 (+ 516us) client_negotiation.cc:240] Negotiation successful
0928 14:44:29.942858 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":53,"threads_started":1}
I20260928 14:44:29.943415 20997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "837f7511dca143ba8e45d34a2d2f1d60" candidate_uuid: "7c54f609458642ca9897be82622f195b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7f4643e91ff415db4c0f5db4e9e0e87" is_pre_election: true
I20260928 14:44:29.943583 20997 raft_consensus.cc:2454] T 837f7511dca143ba8e45d34a2d2f1d60 P a7f4643e91ff415db4c0f5db4e9e0e87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c54f609458642ca9897be82622f195b in term 0.
I20260928 14:44:29.943940 20806 leader_election.cc:302] T 837f7511dca143ba8e45d34a2d2f1d60 P 7c54f609458642ca9897be82622f195b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c54f609458642ca9897be82622f195b, a7f4643e91ff415db4c0f5db4e9e0e87; no voters:
I20260928 14:44:29.944089 21079 raft_consensus.cc:2781] T 837f7511dca143ba8e45d34a2d2f1d60 P 7c54f609458642ca9897be82622f195b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:44:29.944131 21079 raft_consensus.cc:482] T 837f7511dca143ba8e45d34a2d2f1d60 P 7c54f609458642ca9897be82622f195b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:44:29.944151 21079 raft_consensus.cc:3037] T 837f7511dca143ba8e45d34a2d2f1d60 P 7c54f609458642ca9897be82622f195b [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:44:29.945084 21079 raft_consensus.cc:504] T 837f7511dca143ba8e45d34a2d2f1d60 P 7c54f609458642ca9897be82622f195b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7f4643e91ff415db4c0f5db4e9e0e87" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39301 } } peers { permanent_uuid: "7c54f609458642ca9897be82622f195b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 43975 } } peers { permanent_uuid: "1c125a802e6a447cb147f24e2f2f7b2d" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 43827 } }
I20260928 14:44:29.945214 21079 leader_election.cc:288] T 837f7511dca143ba8e45d34a2d2f1d60 P 7c54f609458642ca9897be82622f195b [CANDIDATE]: Term 1 election: Requested vote from peers a7f4643e91ff415db4c0f5db4e9e0e87 (127.9.33.67:39301), 1c125a802e6a447cb147f24e2f2f7b2d (127.9.33.65:43827)
I20260928 14:44:29.945552 20997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "837f7511dca143ba8e45d34a2d2f1d60" candidate_uuid: "7c54f609458642ca9897be82622f195b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7f4643e91ff415db4c0f5db4e9e0e87"
I20260928 14:44:29.945626 20997 raft_consensus.cc:3037] T 837f7511dca143ba8e45d34a2d2f1d60 P a7f4643e91ff415db4c0f5db4e9e0e87 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:44:29.946445 20997 raft_consensus.cc:2454] T 837f7511dca143ba8e45d34a2d2f1d60 P a7f4643e91ff415db4c0f5db4e9e0e87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c54f609458642ca9897be82622f195b in term 1.
I20260928 14:44:29.946762 20806 leader_election.cc:302] T 837f7511dca143ba8e45d34a2d2f1d60 P 7c54f609458642ca9897be82622f195b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c54f609458642ca9897be82622f195b, a7f4643e91ff415db4c0f5db4e9e0e87; no voters:
I20260928 14:44:29.946878 21079 raft_consensus.cc:2781] T 837f7511dca143ba8e45d34a2d2f1d60 P 7c54f609458642ca9897be82622f195b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:44:29.947023 21079 raft_consensus.cc:686] T 837f7511dca143ba8e45d34a2d2f1d60 P 7c54f609458642ca9897be82622f195b [term 1 LEADER]: Becoming Leader. State: Replica: 7c54f609458642ca9897be82622f195b, State: Running, Role: LEADER
I20260928 14:44:29.947110 21079 consensus_queue.cc:237] T 837f7511dca143ba8e45d34a2d2f1d60 P 7c54f609458642ca9897be82622f195b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7f4643e91ff415db4c0f5db4e9e0e87" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39301 } } peers { permanent_uuid: "7c54f609458642ca9897be82622f195b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 43975 } } peers { permanent_uuid: "1c125a802e6a447cb147f24e2f2f7b2d" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 43827 } }
I20260928 14:44:29.948060 20569 catalog_manager.cc:5495] T 837f7511dca143ba8e45d34a2d2f1d60 P 7c54f609458642ca9897be82622f195b reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c54f609458642ca9897be82622f195b (127.9.33.66). New cstate: current_term: 1 leader_uuid: "7c54f609458642ca9897be82622f195b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7f4643e91ff415db4c0f5db4e9e0e87" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c54f609458642ca9897be82622f195b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 43975 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c125a802e6a447cb147f24e2f2f7b2d" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 43827 } health_report { overall_health: UNKNOWN } } }
I20260928 14:44:29.950423 21070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:44:29.938362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:56031 (local address 127.9.33.65:43827)
0928 14:44:29.938419 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0928 14:44:29.938421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:44:29.941400 (+ 2979us) server_negotiation.cc:407] Connection header received
0928 14:44:29.941458 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:44:29.941459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:44:29.941475 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:44:29.941547 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:44:29.946726 (+ 5179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:44:29.947432 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:44:29.949235 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:44:29.949962 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:44:29.950049 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:44:29.950064 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0928 14:44:29.950302 (+ 238us) server_negotiation.cc:299] Negotiation successful
0928 14:44:29.950357 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260928 14:44:29.950826 21080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:44:29.938071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:43827 (local address 127.9.33.66:56031)
0928 14:44:29.941335 (+ 3264us) negotiation.cc:107] Waiting for socket to connect
0928 14:44:29.941346 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:44:29.941399 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:44:29.941610 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:44:29.941612 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:44:29.941614 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:44:29.941691 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:44:29.941692 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:44:29.947592 (+ 5900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:44:29.947593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:44:29.949106 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:44:29.949111 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:44:29.950105 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:44:29.950107 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:44:29.950200 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:44:29.950206 (+ 6us) client_negotiation.cc:769] Sending connection context
0928 14:44:29.950681 (+ 475us) client_negotiation.cc:240] Negotiation successful
0928 14:44:29.950730 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3249,"thread_start_us":45,"threads_started":1}
I20260928 14:44:29.951290 20731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "837f7511dca143ba8e45d34a2d2f1d60" candidate_uuid: "7c54f609458642ca9897be82622f195b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c125a802e6a447cb147f24e2f2f7b2d" is_pre_election: true
I20260928 14:44:29.951443 20731 raft_consensus.cc:2454] T 837f7511dca143ba8e45d34a2d2f1d60 P 1c125a802e6a447cb147f24e2f2f7b2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c54f609458642ca9897be82622f195b in term 0.
I20260928 14:44:29.951545 20732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "837f7511dca143ba8e45d34a2d2f1d60" candidate_uuid: "7c54f609458642ca9897be82622f195b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c125a802e6a447cb147f24e2f2f7b2d"
I20260928 14:44:29.951622 20732 raft_consensus.cc:3037] T 837f7511dca143ba8e45d34a2d2f1d60 P 1c125a802e6a447cb147f24e2f2f7b2d [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:44:29.952760 20732 raft_consensus.cc:2454] T 837f7511dca143ba8e45d34a2d2f1d60 P 1c125a802e6a447cb147f24e2f2f7b2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c54f609458642ca9897be82622f195b in term 1.
I20260928 14:44:30.029642 21069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:44:30.026182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43110 (local address 127.9.33.66:43975)
0928 14:44:30.026261 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0928 14:44:30.026264 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:44:30.026275 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:44:30.026323 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:44:30.026324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:44:30.026352 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:44:30.026438 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:44:30.026606 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:44:30.027376 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:44:30.028503 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:44:30.028800 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:44:30.028856 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:44:30.029038 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:44:30.029192 (+ 154us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:44:30.029264 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 14:44:30.029515 (+ 251us) server_negotiation.cc:299] Negotiation successful
0928 14:44:30.029538 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260928 14:44:30.032151 20732 raft_consensus.cc:1261] T 837f7511dca143ba8e45d34a2d2f1d60 P 1c125a802e6a447cb147f24e2f2f7b2d [term 1 FOLLOWER]: Refusing update from remote peer 7c54f609458642ca9897be82622f195b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:44:30.032600 21082 consensus_queue.cc:1035] T 837f7511dca143ba8e45d34a2d2f1d60 P 7c54f609458642ca9897be82622f195b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c125a802e6a447cb147f24e2f2f7b2d" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 43827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:44:30.033231 20997 raft_consensus.cc:1261] T 837f7511dca143ba8e45d34a2d2f1d60 P a7f4643e91ff415db4c0f5db4e9e0e87 [term 1 FOLLOWER]: Refusing update from remote peer 7c54f609458642ca9897be82622f195b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:44:30.033564 21082 consensus_queue.cc:1035] T 837f7511dca143ba8e45d34a2d2f1d60 P 7c54f609458642ca9897be82622f195b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7f4643e91ff415db4c0f5db4e9e0e87" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 39301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:44:30.036159 21089 mvcc.cc:205] Tried to move back new op lower bound from 7334324920448450560 to 7334324920104087552. Current Snapshot: MvccSnapshot[applied={T|T < 7334324920448450560}]
I20260928 14:44:30.046186 20569 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:44:30.046298 20569 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:44:30.047168 20569 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fcf77c3247174723a615a7844e078758: Sending DeleteTablet for 3 replicas of tablet 837f7511dca143ba8e45d34a2d2f1d60
I20260928 14:44:30.047439 21087 mvcc.cc:205] Tried to move back new op lower bound from 7334324920448450560 to 7334324920104087552. Current Snapshot: MvccSnapshot[applied={T|T < 7334324920448450560}]
I20260928 14:44:30.047907 20987 tablet_service.cc:1514] Processing DeleteTablet for tablet 837f7511dca143ba8e45d34a2d2f1d60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:44:30 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:44678
I20260928 14:44:30.048162 20710 tablet_service.cc:1514] Processing DeleteTablet for tablet 837f7511dca143ba8e45d34a2d2f1d60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:44:30 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:54998
I20260928 14:44:30.048460 20843 tablet_service.cc:1514] Processing DeleteTablet for tablet 837f7511dca143ba8e45d34a2d2f1d60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:44:30 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:43102
I20260928 14:44:30.048977 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 20650
I20260928 14:44:30.052160 21095 tablet_replica.cc:330] T 837f7511dca143ba8e45d34a2d2f1d60 P a7f4643e91ff415db4c0f5db4e9e0e87: stopping tablet replica
I20260928 14:44:30.052297 21095 raft_consensus.cc:2229] T 837f7511dca143ba8e45d34a2d2f1d60 P a7f4643e91ff415db4c0f5db4e9e0e87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:44:30.052433 21095 raft_consensus.cc:2258] T 837f7511dca143ba8e45d34a2d2f1d60 P a7f4643e91ff415db4c0f5db4e9e0e87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:44:30.054315 21095 ts_tablet_manager.cc:1905] T 837f7511dca143ba8e45d34a2d2f1d60 P a7f4643e91ff415db4c0f5db4e9e0e87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:44:30.057449 21097 tablet_replica.cc:330] T 837f7511dca143ba8e45d34a2d2f1d60 P 7c54f609458642ca9897be82622f195b: stopping tablet replica
I20260928 14:44:30.057663 21097 raft_consensus.cc:2229] T 837f7511dca143ba8e45d34a2d2f1d60 P 7c54f609458642ca9897be82622f195b [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:44:30.057898 21097 raft_consensus.cc:2258] T 837f7511dca143ba8e45d34a2d2f1d60 P 7c54f609458642ca9897be82622f195b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:44:30.058455 21097 ts_tablet_manager.cc:1905] T 837f7511dca143ba8e45d34a2d2f1d60 P 7c54f609458642ca9897be82622f195b: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 28 14:44:30 dist-test-slave-pv0w krb5kdc[20092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606669, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for HTTP/127.9.33.84@KRBTEST.COM
I20260928 14:44:30.063109 21097 ts_tablet_manager.cc:1918] T 837f7511dca143ba8e45d34a2d2f1d60 P 7c54f609458642ca9897be82622f195b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:44:30.063277 21097 log.cc:1196] T 837f7511dca143ba8e45d34a2d2f1d60 P 7c54f609458642ca9897be82622f195b: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-1/wal/wals/837f7511dca143ba8e45d34a2d2f1d60
I20260928 14:44:30.063652 21097 ts_tablet_manager.cc:1939] T 837f7511dca143ba8e45d34a2d2f1d60 P 7c54f609458642ca9897be82622f195b: Deleting consensus metadata
I20260928 14:44:30.064270 20555 catalog_manager.cc:4841] TS 7c54f609458642ca9897be82622f195b (127.9.33.66:43975): tablet 837f7511dca143ba8e45d34a2d2f1d60 (table test-table [id=c104fab11b35491cadfc2632c297d268]) successfully deleted
I20260928 14:44:30.065002 21095 ts_tablet_manager.cc:1918] T 837f7511dca143ba8e45d34a2d2f1d60 P a7f4643e91ff415db4c0f5db4e9e0e87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:44:30.065112 21095 log.cc:1196] T 837f7511dca143ba8e45d34a2d2f1d60 P a7f4643e91ff415db4c0f5db4e9e0e87: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790606561545179-9349-0/minicluster-data/ts-2/wal/wals/837f7511dca143ba8e45d34a2d2f1d60
I20260928 14:44:30.065637 21095 ts_tablet_manager.cc:1939] T 837f7511dca143ba8e45d34a2d2f1d60 P a7f4643e91ff415db4c0f5db4e9e0e87: Deleting consensus metadata
I20260928 14:44:30.066141 20553 catalog_manager.cc:4841] TS a7f4643e91ff415db4c0f5db4e9e0e87 (127.9.33.67:39301): tablet 837f7511dca143ba8e45d34a2d2f1d60 (table test-table [id=c104fab11b35491cadfc2632c297d268]) successfully deleted
W20260928 14:44:30.069676 20553 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.33.65:43827 (error 108)
I20260928 14:44:30.070689 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 20789
I20260928 14:44:30.075981 21065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:44:30.075696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:43827 (local address 127.0.0.1:55004)
0928 14:44:30.075879 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0928 14:44:30.075936 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.65:43827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
W20260928 14:44:30.076311 20553 catalog_manager.cc:4584] TS 1c125a802e6a447cb147f24e2f2f7b2d (127.9.33.65:43827): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 837f7511dca143ba8e45d34a2d2f1d60: Network error: Client connection negotiation failed: client connection to 127.9.33.65:43827: connect: Connection refused (error 111)
I20260928 14:44:30.088016 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 20924
I20260928 14:44:30.094278 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 20537
I20260928 14:44:30.101634 9349 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260928 14:44:30.446599 9349 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260928 14:44:30.446717 9349 mini_ranger.cc:66] Stopping Ranger...
I20260928 14:44:30.792165 9349 mini_ranger.cc:68] Stopped Ranger
2026-09-28 14:44:30.792 UTC [20122] LOG: received smart shutdown request
2026-09-28 14:44:30.796 UTC [20122] LOG: background worker "logical replication launcher" (PID 20131) exited with exit code 1
2026-09-28 14:44:30.796 UTC [20126] LOG: shutting down
2026-09-28 14:44:31.012 UTC [20122] LOG: database system is shut down
2026-09-28T14:44:31Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50909 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:44:31 dist-test-slave-pv0w krb5kdc[21108](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:44:31 dist-test-slave-pv0w krb5kdc[21108](info): set up 2 sockets
Sep 28 14:44:31 dist-test-slave-pv0w krb5kdc[21108](info): commencing operation
krb5kdc: starting...
W20260928 14:44:32.164379 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:44:32 dist-test-slave-pv0w krb5kdc[21108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:44:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:44:32Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.9.33.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.9.33.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.9.33.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/rangeradmin_127.9.33.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.9.33.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/rangeradmin_127.9.33.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.9.33.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.9.33.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.9.33.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/rangerlookup_127.9.33.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.9.33.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/rangerlookup_127.9.33.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.33.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.9.33.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.33.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/HTTP_127.9.33.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.33.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/HTTP_127.9.33.84@KRBTEST.COM.keytab.
I20260928 14:44:32.343323 9349 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task4TnxN9/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/postgres -l logfile start
2026-09-28 14:44:35.256 UTC [21138] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-28 14:44:35.256 UTC [21138] LOG: listening on IPv4 address "127.9.33.84", port 32869
2026-09-28 14:44:35.260 UTC [21138] LOG: listening on Unix socket "/tmp/.s.PGSQL.32869"
2026-09-28 14:44:35.296 UTC [21141] LOG: database system was shut down at 2026-09-28 14:44:33 UTC
2026-09-28 14:44:35.301 UTC [21138] LOG: database system is ready to accept connections
I20260928 14:44:36.248344 9349 mini_postgres.cc:96] Postgres bound to 32869
2026-09-28 14:44:36.254 UTC [21149] FATAL: database "slave" does not exist
127.9.33.84:32869 - accepting connections
I20260928 14:44:36.255168 9349 mini_ranger.cc:161] Starting Ranger...
I20260928 14:44:36.270936 9349 mini_ranger.cc:84] Created miniranger Postgres user
I20260928 14:44:36.848484 9349 mini_ranger.cc:87] Created ranger Postgres database
I20260928 14:44:36.848577 9349 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ranger-admin
2026-09-28 14:44:37,132 [I] DB FLAVOR :POSTGRES
2026-09-28 14:44:37,133 [I] --------- Verifying Ranger DB connection ---------
2026-09-28 14:44:37,133 [I] Checking connection..
2026-09-28 14:44:37,133 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:32869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-28 14:44:37,393 [I] Checking connection passed.
2026-09-28 14:44:37,393 [I] --------- Verifying version history table ---------
2026-09-28 14:44:37,394 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:32869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 14:44:37,665 [I] Table x_db_version_h does not exist in database ranger
2026-09-28 14:44:37,666 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:32869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 14:44:37,933 [I] Table x_db_version_h does not exist in database ranger
2026-09-28 14:44:37,933 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-28 14:44:37,933 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:32869/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-28 14:44:38.189 UTC [21213] WARNING: there is no transaction in progress
2026-09-28 14:44:38,196 [I] create_dbversion_catalog.sql file imported successfully
2026-09-28 14:44:38,197 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:32869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-28 14:44:38,470 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-28 14:44:38,471 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-28 14:44:38,471 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:32869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-28 14:44:38,722 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:32869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-28 14:44:38,980 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:32869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-pv0w.c.gcp-upstream.internal','N') ;"
2026-09-28 14:44:39,221 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:32869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-28 14:44:39,499 [I] Table x_portal_user does not exist in database ranger
2026-09-28 14:44:39,500 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:32869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-28 14:44:39,749 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-28 14:44:39,750 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:32869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-28 14:44:40,016 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:32869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-28 14:44:40,290 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-28 14:44:40,290 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:32869/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-28 14:44:41.390 UTC [21321] WARNING: there is no transaction in progress
2026-09-28 14:44:41.410 UTC [21321] WARNING: there is no transaction in progress
2026-09-28 14:44:41.429 UTC [21321] WARNING: there is no transaction in progress
2026-09-28 14:44:41.448 UTC [21321] WARNING: there is no transaction in progress
2026-09-28 14:44:41.466 UTC [21321] WARNING: there is no transaction in progress
2026-09-28 14:44:41.489 UTC [21321] WARNING: there is no transaction in progress
2026-09-28 14:44:41.554 UTC [21321] WARNING: there is no transaction in progress
2026-09-28 14:44:41.562 UTC [21321] WARNING: there is no transaction in progress
2026-09-28 14:44:41.577 UTC [21321] WARNING: there is no transaction in progress
2026-09-28 14:44:41.588 UTC [21321] WARNING: there is no transaction in progress
2026-09-28 14:44:41.600 UTC [21321] WARNING: there is no transaction in progress
2026-09-28 14:44:41.615 UTC [21321] WARNING: there is no transaction in progress
2026-09-28 14:44:41.626 UTC [21321] WARNING: there is no transaction in progress
2026-09-28 14:44:41.634 UTC [21321] WARNING: there is no transaction in progress
2026-09-28 14:44:41.643 UTC [21321] WARNING: there is no transaction in progress
2026-09-28 14:44:41.651 UTC [21321] WARNING: there is no transaction in progress
2026-09-28 14:44:42,459 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-28 14:44:42,459 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:32869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-28 14:44:42,704 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-28 14:44:42,704 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:32869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-28 14:44:42,960 [I] Table x_portal_user already exists in database 'ranger'
2026-09-28 14:44:42,960 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:32869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-28 14:44:43,220 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-28 14:44:43,220 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:32869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-28 14:44:43,469 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:32869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-28 14:44:43,719 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:32869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-pv0w.c.gcp-upstream.internal';"
2026-09-28 14:44:43,975 [I] CORE_DB_SCHEMA import status has been updated
2026-09-28 14:44:43,975 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:32869/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-28 14:44:44,234 [I] DB_PATCHES have already been applied
I20260928 14:44:44.240528 9349 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ranger-admin:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task4TnxN9/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260928 14:44:44.240617 9349 mini_ranger.cc:193] Using host: 127.9.33.84
I20260928 14:44:44.243855 9349 mini_ranger.cc:227] Ranger admin URL: http://127.9.33.84:39813
Sep 28, 2026 2:44:44 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 28, 2026 2:44:44 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 28, 2026 2:44:44 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ranger-admin/ews/webapp
Sep 28, 2026 2:44:44 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 28, 2026 2:44:44 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 28, 2026 2:44:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ranger-admin/ews/webapp].
Sep 28, 2026 2:44:45 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260928 14:44:45.380026 9349 mini_ranger.cc:160] Time spent starting Ranger: real 9.125s user 0.000s sys 0.010s
Sep 28, 2026 2:45:06 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 28, 2026 2:45:06 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 28, 2026 2:45:06 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 28, 2026 2:45:06 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 28, 2026 2:45:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 28, 2026 2:45:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 28, 2026 2:45:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 28, 2026 2:45:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 28, 2026 2:45:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 28, 2026 2:45:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 28, 2026 2:45:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 28, 2026 2:45:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 28, 2026 2:45:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 28, 2026 2:45:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 28, 2026 2:45:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 28, 2026 2:45:06 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 28, 2026 2:45:06 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 28, 2026 2:45:07 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260928 14:45:08.247020 9349 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.9.33.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.9.33.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.9.33.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/rangerkms_127.9.33.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.9.33.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/rangerkms_127.9.33.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.33.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.9.33.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.33.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/HTTP_127.9.33.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.33.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/HTTP_127.9.33.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 28 14:45:08 dist-test-slave-pv0w krb5kdc[21108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606708, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260928 14:45:08.338847 9349 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260928 14:45:08.348968 9349 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260928 14:45:09.372177 9349 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260928 14:45:09.372274 9349 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ranger-kms
I20260928 14:45:09.372298 9349 mini_ranger_kms.cc:227] Using postgres at 127.9.33.84:32869
2026-09-28 14:45:09,549 [I] DB FLAVOR :POSTGRES
2026-09-28 14:45:09,550 [I] --------- Verifying Ranger DB connection ---------
2026-09-28 14:45:09,550 [I] Checking connection
2026-09-28 14:45:09,550 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:32869/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-28 14:45:09,803 [I] connection success
2026-09-28 14:45:09,803 [I] --------- Verifying Ranger DB tables ---------
2026-09-28 14:45:09,803 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-28 14:45:09,803 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:32869/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-28 14:45:10,060 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-28 14:45:10,060 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-28 14:45:10,060 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-28 14:45:10,060 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.33.84:32869/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-28 14:45:10,332 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260928 14:45:10.871640 9349 mini_ranger_kms.cc:310] Created kms service
I20260928 14:45:11.100714 9349 mini_ranger_kms.cc:326] Created kudu user
I20260928 14:45:11.150651 9349 mini_ranger_kms.cc:343] Created rangerkms user
Sep 28 14:45:11 dist-test-slave-pv0w krb5kdc[21108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606708, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.9.33.84@KRBTEST.COM
I20260928 14:45:11.605448 9349 mini_ranger_kms.cc:384] Added ranger policy
I20260928 14:45:11.605706 9349 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ranger-kms:/tmp/dist-test-task4TnxN9/build/release/bin/postgresql.jar:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task4TnxN9/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260928 14:45:11.605734 9349 mini_ranger_kms.cc:240] Using host: 127.9.33.84
I20260928 14:45:11.609359 9349 mini_ranger_kms.cc:276] Ranger KMS PID: 21504
I20260928 14:45:11.609443 9349 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.9.33.84:36395
Sep 28, 2026 2:45:12 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 28, 2026 2:45:12 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 28, 2026 2:45:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 28, 2026 2:45:12 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 28, 2026 2:45:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ranger-kms/ews/webapp].
Sep 28, 2026 2:45:13 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Sep 28 14:45:13 dist-test-slave-pv0w krb5kdc[21108](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790606713, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.33.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:45:13.952447 9349 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.614s user 0.001s sys 0.158s
I20260928 14:45:13.952556 9349 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260928 14:45:13.952621 9349 mini_ranger_kms.cc:401] 127.9.33.84:36395/kms/v1/keys
Sep 28 14:45:14 dist-test-slave-pv0w krb5kdc[21108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606713, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.33.84@KRBTEST.COM for HTTP/127.9.33.84@KRBTEST.COM
Sep 28 14:45:14 dist-test-slave-pv0w krb5kdc[21108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606713, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.33.84@KRBTEST.COM for HTTP/127.9.33.84@KRBTEST.COM
Sep 28 14:45:17 dist-test-slave-pv0w krb5kdc[21108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.9.33.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
I20260928 14:45:17.088186 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:38143
--webserver_interface=127.9.33.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:45957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:38143
--ranger_config_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.33.84:36395/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:45:17.181456 21554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:45:17.181716 21554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:45:17.181757 21554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:45:17.183051 21554 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:45:17.183135 21554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:45:17.183159 21554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:45:17.183192 21554 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:45:17.183220 21554 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:45:17.185068 21554 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:45957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/master-0/wal
--ranger_kms_url=127.9.33.84:36395/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:38143
--ranger_config_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:38143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:45:17.185375 21554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:45:17.185624 21554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:45:17.190914 21560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:17.191067 21562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:17.190922 21559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:45:17.191296 21554 server_base.cc:1034] running on GCE node
I20260928 14:45:17.191632 21554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:45:17.192010 21554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:45:17.193181 21554 hybrid_clock.cc:648] HybridClock initialized: now 1790606717193156 us; error 41 us; skew 500 ppm
Sep 28 14:45:17 dist-test-slave-pv0w krb5kdc[21108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606717, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:45:17.196043 21554 init.cc:377] Logged in from keytab as kudu/127.9.33.126@KRBTEST.COM (short username kudu)
I20260928 14:45:17.196606 21554 webserver.cc:460] Webserver started at http://127.9.33.126:46809/ using document root <none> and password file <none>
I20260928 14:45:17.196871 21554 fs_manager.cc:359] Metadata directory not provided
I20260928 14:45:17.196933 21554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:45:17.197105 21554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 14:45:17 dist-test-slave-pv0w krb5kdc[21108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606717, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for HTTP/127.9.33.84@KRBTEST.COM
I20260928 14:45:17.259240 21554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "84f7c09ac7624bcaa7e2c2512e47c346"
format_stamp: "Formatted at 2026-09-28 14:45:17 on dist-test-slave-pv0w"
server_key: "b17deaf4fcbe8ee31e9070ffb6f2cdc7"
server_key_iv: "af5a75a6c64c51fe20d2029c12ff0731"
server_key_version: "kuduclusterkey@0"
I20260928 14:45:17.259707 21554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "84f7c09ac7624bcaa7e2c2512e47c346"
format_stamp: "Formatted at 2026-09-28 14:45:17 on dist-test-slave-pv0w"
server_key: "b17deaf4fcbe8ee31e9070ffb6f2cdc7"
server_key_iv: "af5a75a6c64c51fe20d2029c12ff0731"
server_key_version: "kuduclusterkey@0"
I20260928 14:45:17.261273 21554 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:45:17.274997 21569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:17.275278 21554 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:45:17.275379 21554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "84f7c09ac7624bcaa7e2c2512e47c346"
format_stamp: "Formatted at 2026-09-28 14:45:17 on dist-test-slave-pv0w"
server_key: "b17deaf4fcbe8ee31e9070ffb6f2cdc7"
server_key_iv: "af5a75a6c64c51fe20d2029c12ff0731"
server_key_version: "kuduclusterkey@0"
I20260928 14:45:17.275489 21554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:45:17.298322 21554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:45:17.300750 21554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:45:17.300937 21554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:45:17.306537 21554 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:38143
I20260928 14:45:17.306563 21622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:38143 every 8 connection(s)
I20260928 14:45:17.307055 21554 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:45:17.307945 21623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:45:17.311221 21623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346: Bootstrap starting.
I20260928 14:45:17.312022 21623 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346: Neither blocks nor log segments found. Creating new log.
I20260928 14:45:17.312347 21623 log.cc:824] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346: Log is configured to *not* fsync() on all Append() calls
I20260928 14:45:17.313189 21623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346: No bootstrap required, opened a new log
I20260928 14:45:17.315784 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 21554
I20260928 14:45:17.315936 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:45:17.316133 9349 external_mini_cluster.cc:1428] Setting key 9b57c0ded694a4c934ba5ad59cd8e7ed
I20260928 14:45:17.316205 21623 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84f7c09ac7624bcaa7e2c2512e47c346" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38143 } }
I20260928 14:45:17.316432 21623 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:45:17.316468 21623 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84f7c09ac7624bcaa7e2c2512e47c346, State: Initialized, Role: FOLLOWER
I20260928 14:45:17.316661 21623 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84f7c09ac7624bcaa7e2c2512e47c346" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38143 } }
I20260928 14:45:17.316855 21623 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:45:17.317088 21623 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:45:17.317157 21623 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:45:17.318063 21623 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84f7c09ac7624bcaa7e2c2512e47c346" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38143 } }
I20260928 14:45:17.318189 21623 leader_election.cc:302] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84f7c09ac7624bcaa7e2c2512e47c346; no voters:
I20260928 14:45:17.318394 21623 leader_election.cc:288] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:45:17.318454 21628 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:45:17.318711 21628 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346 [term 1 LEADER]: Becoming Leader. State: Replica: 84f7c09ac7624bcaa7e2c2512e47c346, State: Running, Role: LEADER
I20260928 14:45:17.318737 21623 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:45:17.318893 21628 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84f7c09ac7624bcaa7e2c2512e47c346" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38143 } }
I20260928 14:45:17.319209 21628 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84f7c09ac7624bcaa7e2c2512e47c346. Latest consensus state: current_term: 1 leader_uuid: "84f7c09ac7624bcaa7e2c2512e47c346" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84f7c09ac7624bcaa7e2c2512e47c346" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38143 } } }
I20260928 14:45:17.319267 21628 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346 [sys.catalog]: This master's current role is: LEADER
I20260928 14:45:17.319981 21635 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:45:17.320289 21623 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260928 14:45:17.320505 21635 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:45:17.320776 21629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84f7c09ac7624bcaa7e2c2512e47c346" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84f7c09ac7624bcaa7e2c2512e47c346" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38143 } } }
I20260928 14:45:17.320904 21629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346 [sys.catalog]: This master's current role is: LEADER
Sep 28 14:45:17 dist-test-slave-pv0w krb5kdc[21108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:45:17.333948 21627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:17.316798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57254 (local address 127.9.33.126:38143)
0928 14:45:17.317001 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:17.317013 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:17.317045 (+ 32us) server_negotiation.cc:407] Connection header received
0928 14:45:17.317195 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:17.317204 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:17.317267 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:17.317438 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:17.318080 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:17.319660 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:17.320865 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:17.324734 (+ 3869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:17.324997 (+ 263us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:17.328485 (+ 3488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:17.328492 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:17.328492 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:17.328508 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:17.331515 (+ 3007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:17.332061 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:17.332065 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:17.332066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:17.332200 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:17.332417 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:17.332418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:17.332418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:17.332790 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:17.332962 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:17.333535 (+ 573us) server_negotiation.cc:299] Negotiation successful
0928 14:45:17.333682 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":94,"threads_started":1}
I20260928 14:45:17.340327 21635 catalog_manager.cc:1347] Generated new cluster ID: 062e8a252f4449089ba40ca2022a690a
I20260928 14:45:17.340402 21635 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:45:17.352506 21635 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:45:17.353379 21635 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:45:17.363308 21635 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346: Generated new TSK 0
I20260928 14:45:17.363754 21635 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.33.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab.
Entry for principal kudu/127.9.33.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab.
I20260928 14:45:18.071653 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.33.84:36395/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.65:0
--local_ip_for_outbound_sockets=127.9.33.65
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:38143
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:45957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:45:18.178548 21668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:45:18.178803 21668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:45:18.178848 21668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:45:18.180287 21668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:45:18.180353 21668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:45:18.180405 21668 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.65
I20260928 14:45:18.182134 21668 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:45957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.9.33.84:36395/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:38143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.65
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:45:18.182410 21668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:45:18.182734 21668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:45:18.189411 21673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:18.189440 21674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:18.189433 21676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:45:18.189913 21668 server_base.cc:1034] running on GCE node
I20260928 14:45:18.190145 21668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:45:18.190459 21668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:45:18.191656 21668 hybrid_clock.cc:648] HybridClock initialized: now 1790606718191640 us; error 34 us; skew 500 ppm
Sep 28 14:45:18 dist-test-slave-pv0w krb5kdc[21108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606718, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:45:18.195240 21668 init.cc:377] Logged in from keytab as kudu/127.9.33.65@KRBTEST.COM (short username kudu)
I20260928 14:45:18.195914 21668 webserver.cc:460] Webserver started at http://127.9.33.65:34625/ using document root <none> and password file <none>
I20260928 14:45:18.196223 21668 fs_manager.cc:359] Metadata directory not provided
I20260928 14:45:18.196298 21668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:45:18.196484 21668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 14:45:18 dist-test-slave-pv0w krb5kdc[21108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606718, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for HTTP/127.9.33.84@KRBTEST.COM
I20260928 14:45:18.212517 21668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-0/data/instance:
uuid: "c1d06384fce2442aa53ee42518d4da95"
format_stamp: "Formatted at 2026-09-28 14:45:18 on dist-test-slave-pv0w"
server_key: "d61fa51ef5839e42328dad656a90197f"
server_key_iv: "71c5061ce1738d2ce785ba78e376fb2e"
server_key_version: "kuduclusterkey@0"
I20260928 14:45:18.213132 21668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance:
uuid: "c1d06384fce2442aa53ee42518d4da95"
format_stamp: "Formatted at 2026-09-28 14:45:18 on dist-test-slave-pv0w"
server_key: "d61fa51ef5839e42328dad656a90197f"
server_key_iv: "71c5061ce1738d2ce785ba78e376fb2e"
server_key_version: "kuduclusterkey@0"
I20260928 14:45:18.217309 21668 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.000s
I20260928 14:45:18.236935 21685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:18.237205 21668 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:45:18.237313 21668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-0/wal
uuid: "c1d06384fce2442aa53ee42518d4da95"
format_stamp: "Formatted at 2026-09-28 14:45:18 on dist-test-slave-pv0w"
server_key: "d61fa51ef5839e42328dad656a90197f"
server_key_iv: "71c5061ce1738d2ce785ba78e376fb2e"
server_key_version: "kuduclusterkey@0"
I20260928 14:45:18.237419 21668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:45:18.299903 21668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:45:18.302837 21668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:45:18.303012 21668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:45:18.303263 21668 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:45:18.303637 21668 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:45:18.303685 21668 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:18.303746 21668 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:45:18.303771 21668 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:18.321610 21668 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.65:37539
I20260928 14:45:18.322154 21668 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
I20260928 14:45:18.322947 21801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.65:37539 every 8 connection(s)
I20260928 14:45:18.325114 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 21668
I20260928 14:45:18.325217 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance
I20260928 14:45:18.325428 9349 external_mini_cluster.cc:1428] Setting key fc358f34dfa9b46818a7874f40ba3355
Sep 28 14:45:18 dist-test-slave-pv0w krb5kdc[21108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606718, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.33.66@KRBTEST.COM; defaulting to no policy
I20260928 14:45:18.354866 21804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:18.323761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:56325 (local address 127.9.33.126:38143)
0928 14:45:18.323917 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:18.323920 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:18.324196 (+ 276us) server_negotiation.cc:407] Connection header received
0928 14:45:18.331599 (+ 7403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:18.331601 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:18.331621 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:18.331775 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:18.339899 (+ 8124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:18.340755 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:18.342613 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:18.342954 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:18.343002 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:18.345946 (+ 2944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:18.345949 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:18.345949 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:18.345958 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:18.348057 (+ 2099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:18.348658 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:18.348661 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:18.348661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:18.348733 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:18.353930 (+ 5197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:18.353932 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:18.353932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:18.354111 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:18.354172 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:18.354605 (+ 433us) server_negotiation.cc:299] Negotiation successful
0928 14:45:18.354720 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":89,"threads_started":1}
I20260928 14:45:18.355859 21805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:18.323709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38143 (local address 127.9.33.65:56325)
0928 14:45:18.324052 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:18.324102 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:18.324827 (+ 725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:18.331991 (+ 7164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:18.331998 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:18.332358 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:45:18.332696 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:18.332704 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:18.340950 (+ 8246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:18.340951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:18.342479 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:18.342485 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:18.343125 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:18.343126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:18.343302 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:18.344016 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:45:18.344028 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:45:18.345797 (+ 1769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:45:18.348181 (+ 2384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:18.348193 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:18.348196 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:18.348551 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:18.353506 (+ 4955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:18.353508 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:18.353508 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:18.353788 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:18.354212 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:45:18.354222 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:45:18.354443 (+ 221us) client_negotiation.cc:769] Sending connection context
0928 14:45:18.355285 (+ 842us) client_negotiation.cc:240] Negotiation successful
0928 14:45:18.355496 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":104,"threads_started":1}
I20260928 14:45:18.356657 21802 heartbeater.cc:347] Connected to a master server at 127.9.33.126:38143
I20260928 14:45:18.356771 21802 heartbeater.cc:464] Registering TS with master...
I20260928 14:45:18.356978 21802 heartbeater.cc:511] Master 127.9.33.126:38143 requested a full tablet report, sending...
I20260928 14:45:18.357661 21587 ts_manager.cc:194] Registered new tserver with Master: c1d06384fce2442aa53ee42518d4da95 (127.9.33.65:37539)
I20260928 14:45:18.358593 21587 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.65@KRBTEST.COM'} at 127.9.33.65:56325
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab.
Entry for principal kudu/127.9.33.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab.
I20260928 14:45:18.387657 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.33.84:36395/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.66:0
--local_ip_for_outbound_sockets=127.9.33.66
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:38143
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:45957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 14:45:18 dist-test-slave-pv0w krb5kdc[21108](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790606718, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260928 14:45:18.499522 21808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:45:18.499941 21808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:45:18.500034 21808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:45:18.501968 21808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:45:18.502038 21808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:45:18.502085 21808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.66
I20260928 14:45:18.504597 21808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:45957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.9.33.84:36395/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:38143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.66
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:45:18.504889 21808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:45:18.505221 21808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:45:18.511991 21817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:45:18.512198 21808 server_base.cc:1034] running on GCE node
W20260928 14:45:18.512406 21815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:18.512420 21814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:45:18.512804 21808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:45:18.513098 21808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:45:18.514277 21808 hybrid_clock.cc:648] HybridClock initialized: now 1790606718514240 us; error 46 us; skew 500 ppm
Sep 28 14:45:18 dist-test-slave-pv0w krb5kdc[21108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606718, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:45:18.517639 21808 init.cc:377] Logged in from keytab as kudu/127.9.33.66@KRBTEST.COM (short username kudu)
I20260928 14:45:18.518239 21808 webserver.cc:460] Webserver started at http://127.9.33.66:41679/ using document root <none> and password file <none>
I20260928 14:45:18.518528 21808 fs_manager.cc:359] Metadata directory not provided
I20260928 14:45:18.518601 21808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:45:18.518748 21808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 14:45:18 dist-test-slave-pv0w krb5kdc[21108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606718, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for HTTP/127.9.33.84@KRBTEST.COM
I20260928 14:45:18.537596 21808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-1/data/instance:
uuid: "a7f71b99321a4396bf449f3adbab4ce2"
format_stamp: "Formatted at 2026-09-28 14:45:18 on dist-test-slave-pv0w"
server_key: "e5f1cefba0567a312f7c950b3997c5f7"
server_key_iv: "dcb45ebf36570fbe247d3feaa388873d"
server_key_version: "kuduclusterkey@0"
I20260928 14:45:18.538084 21808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance:
uuid: "a7f71b99321a4396bf449f3adbab4ce2"
format_stamp: "Formatted at 2026-09-28 14:45:18 on dist-test-slave-pv0w"
server_key: "e5f1cefba0567a312f7c950b3997c5f7"
server_key_iv: "dcb45ebf36570fbe247d3feaa388873d"
server_key_version: "kuduclusterkey@0"
I20260928 14:45:18.540481 21808 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.002s
I20260928 14:45:18.575888 21824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:18.576210 21808 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.001s sys 0.000s
I20260928 14:45:18.576324 21808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-1/wal
uuid: "a7f71b99321a4396bf449f3adbab4ce2"
format_stamp: "Formatted at 2026-09-28 14:45:18 on dist-test-slave-pv0w"
server_key: "e5f1cefba0567a312f7c950b3997c5f7"
server_key_iv: "dcb45ebf36570fbe247d3feaa388873d"
server_key_version: "kuduclusterkey@0"
I20260928 14:45:18.576436 21808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:45:18.597622 21808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:45:18.600409 21808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:45:18.600629 21808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:45:18.600908 21808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:45:18.601487 21808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:45:18.601581 21808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:18.601653 21808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:45:18.601711 21808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:18.608311 21808 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.66:41621
I20260928 14:45:18.608848 21808 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
I20260928 14:45:18.610481 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 21808
I20260928 14:45:18.610575 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance
I20260928 14:45:18.610785 9349 external_mini_cluster.cc:1428] Setting key cfdbe4d18a7c501b0556bf2113bdefdd
I20260928 14:45:18.618239 21938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.66:41621 every 8 connection(s)
Sep 28 14:45:18 dist-test-slave-pv0w krb5kdc[21108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606718, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:45:18.632640 21804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:18.619283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:40073 (local address 127.9.33.126:38143)
0928 14:45:18.619353 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:18.619355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:18.621122 (+ 1767us) server_negotiation.cc:407] Connection header received
0928 14:45:18.621911 (+ 789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:18.621913 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:18.621927 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:18.621961 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:18.622893 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:18.623799 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:18.625357 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:18.625669 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:18.625722 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:18.628667 (+ 2945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:18.628669 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:18.628669 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:18.628679 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:18.630698 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:18.631324 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:18.631326 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:18.631327 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:18.631395 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:18.631769 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:18.631770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:18.631771 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:18.631926 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:18.631973 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:18.632435 (+ 462us) server_negotiation.cc:299] Negotiation successful
0928 14:45:18.632539 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260928 14:45:18.633620 21942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:18.619290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38143 (local address 127.9.33.66:40073)
0928 14:45:18.620954 (+ 1664us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:18.621013 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:18.621767 (+ 754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:18.622179 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:18.622184 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:18.622462 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:45:18.622770 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:18.622777 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:18.623926 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:18.623926 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:18.625242 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:18.625245 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:18.625841 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:18.625842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:18.625992 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:18.626691 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:45:18.626703 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:45:18.628525 (+ 1822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:45:18.630838 (+ 2313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:18.630848 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:18.630851 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:18.631222 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:18.631474 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:18.631475 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:18.631475 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:18.631644 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:18.632027 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:45:18.632035 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:45:18.632258 (+ 223us) client_negotiation.cc:769] Sending connection context
0928 14:45:18.633133 (+ 875us) client_negotiation.cc:240] Negotiation successful
0928 14:45:18.633355 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1610,"thread_start_us":119,"threads_started":1}
I20260928 14:45:18.634390 21939 heartbeater.cc:347] Connected to a master server at 127.9.33.126:38143
I20260928 14:45:18.634493 21939 heartbeater.cc:464] Registering TS with master...
I20260928 14:45:18.634722 21939 heartbeater.cc:511] Master 127.9.33.126:38143 requested a full tablet report, sending...
I20260928 14:45:18.635251 21587 ts_manager.cc:194] Registered new tserver with Master: a7f71b99321a4396bf449f3adbab4ce2 (127.9.33.66:41621)
I20260928 14:45:18.635793 21587 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.66@KRBTEST.COM'} at 127.9.33.66:40073
WARNING: no policy specified for kudu/127.9.33.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab.
Entry for principal kudu/127.9.33.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab.
I20260928 14:45:18.663024 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.33.84:36395/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.67:0
--local_ip_for_outbound_sockets=127.9.33.67
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:38143
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:45957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:45:18.792397 21944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:45:18.792791 21944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:45:18.792887 21944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:45:18.794850 21944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:45:18.794982 21944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:45:18.795081 21944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.67
I20260928 14:45:18.797583 21944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:45957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.9.33.84:36395/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:38143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.67
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:45:18.798138 21944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:45:18.798484 21944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:45:18.809765 21950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:18.810104 21952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:18.816357 21949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:45:18.818598 21944 server_base.cc:1034] running on GCE node
I20260928 14:45:18.819046 21944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:45:18.819388 21944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:45:18.826767 21944 hybrid_clock.cc:648] HybridClock initialized: now 1790606718826687 us; error 93 us; skew 500 ppm
Sep 28 14:45:18 dist-test-slave-pv0w krb5kdc[21108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606718, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:45:18.830204 21944 init.cc:377] Logged in from keytab as kudu/127.9.33.67@KRBTEST.COM (short username kudu)
I20260928 14:45:18.830875 21944 webserver.cc:460] Webserver started at http://127.9.33.67:36683/ using document root <none> and password file <none>
I20260928 14:45:18.831174 21944 fs_manager.cc:359] Metadata directory not provided
I20260928 14:45:18.831233 21944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:45:18.831373 21944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 28 14:45:18 dist-test-slave-pv0w krb5kdc[21108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606718, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for HTTP/127.9.33.84@KRBTEST.COM
I20260928 14:45:18.850169 21944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-2/data/instance:
uuid: "07a94cc819234e2d888e84c1c5e3177a"
format_stamp: "Formatted at 2026-09-28 14:45:18 on dist-test-slave-pv0w"
server_key: "c00fa7309762fdd96f5f953bca4324cb"
server_key_iv: "4aaa2ba11688884e89f7cc4f3cbde150"
server_key_version: "kuduclusterkey@0"
I20260928 14:45:18.855383 21944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance:
uuid: "07a94cc819234e2d888e84c1c5e3177a"
format_stamp: "Formatted at 2026-09-28 14:45:18 on dist-test-slave-pv0w"
server_key: "c00fa7309762fdd96f5f953bca4324cb"
server_key_iv: "4aaa2ba11688884e89f7cc4f3cbde150"
server_key_version: "kuduclusterkey@0"
I20260928 14:45:18.861485 21944 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.002s
I20260928 14:45:18.894701 21959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:18.895236 21944 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.001s sys 0.000s
I20260928 14:45:18.895373 21944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-2/wal
uuid: "07a94cc819234e2d888e84c1c5e3177a"
format_stamp: "Formatted at 2026-09-28 14:45:18 on dist-test-slave-pv0w"
server_key: "c00fa7309762fdd96f5f953bca4324cb"
server_key_iv: "4aaa2ba11688884e89f7cc4f3cbde150"
server_key_version: "kuduclusterkey@0"
I20260928 14:45:18.895473 21944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Sep 28 14:45:18 dist-test-slave-pv0w krb5kdc[21108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606718, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for HTTP/127.9.33.84@KRBTEST.COM
I20260928 14:45:18.956202 21944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:45:18.964423 21944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:45:18.964694 21944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:45:18.965016 21944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:45:18.965464 21944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:45:18.965559 21944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:18.965638 21944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:45:18.965700 21944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:18.979379 21944 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.67:36323
I20260928 14:45:18.979991 21944 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
I20260928 14:45:18.984663 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 21944
I20260928 14:45:18.984764 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance
I20260928 14:45:18.984995 9349 external_mini_cluster.cc:1428] Setting key ea258d1abd48d7f34575bf11e0690ee1
I20260928 14:45:18.982921 22072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.67:36323 every 8 connection(s)
Sep 28 14:45:18 dist-test-slave-pv0w krb5kdc[21108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606718, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:45:18.994874 21804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:18.983853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:58017 (local address 127.9.33.126:38143)
0928 14:45:18.983910 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:18.983913 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:18.984312 (+ 399us) server_negotiation.cc:407] Connection header received
0928 14:45:18.985027 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:18.985028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:18.985041 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:18.985071 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:18.985954 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:18.986663 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:18.988332 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:18.988607 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:18.988658 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:18.991342 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:18.991344 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:18.991345 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:18.991352 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:18.993125 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:18.993673 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:18.993675 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:18.993675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:18.993736 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:18.994032 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:18.994033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:18.994033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:18.994192 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:18.994240 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:18.994680 (+ 440us) server_negotiation.cc:299] Negotiation successful
0928 14:45:18.994773 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260928 14:45:18.995754 22075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:18.983530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38143 (local address 127.9.33.67:58017)
0928 14:45:18.984142 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:18.984225 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:18.984905 (+ 680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:18.985280 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:18.985285 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:18.985545 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:45:18.985837 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:18.985843 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:18.986948 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:18.986949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:18.988227 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:18.988230 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:18.988772 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:18.988773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:18.988913 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:18.989524 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:45:18.989535 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:45:18.991224 (+ 1689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:45:18.993247 (+ 2023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:18.993257 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:18.993259 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:18.993584 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:18.993807 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:18.993807 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:18.993807 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:18.993951 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:18.994295 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:45:18.994302 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:45:18.994507 (+ 205us) client_negotiation.cc:769] Sending connection context
0928 14:45:18.995297 (+ 790us) client_negotiation.cc:240] Negotiation successful
0928 14:45:18.995482 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":98,"threads_started":1}
I20260928 14:45:18.996497 22073 heartbeater.cc:347] Connected to a master server at 127.9.33.126:38143
I20260928 14:45:18.996594 22073 heartbeater.cc:464] Registering TS with master...
I20260928 14:45:18.996796 22073 heartbeater.cc:511] Master 127.9.33.126:38143 requested a full tablet report, sending...
I20260928 14:45:18.997270 21587 ts_manager.cc:194] Registered new tserver with Master: 07a94cc819234e2d888e84c1c5e3177a (127.9.33.67:36323)
I20260928 14:45:18.997758 21587 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.67@KRBTEST.COM'} at 127.9.33.67:58017
I20260928 14:45:19.007354 9349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:45:19.008090 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 21668
I20260928 14:45:19.013496 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 21808
I20260928 14:45:19.020633 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 21944
I20260928 14:45:19.026988 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 21554
I20260928 14:45:19.034260 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:38143
--webserver_interface=127.9.33.126
--webserver_port=46809
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:45957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:38143
--ranger_config_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.33.84:36395/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.33.84:36395/kms with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:45:19.138547 22076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:45:19.138816 22076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:45:19.138859 22076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:45:19.140185 22076 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:45:19.140254 22076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:45:19.140280 22076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:45:19.140309 22076 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:45:19.140333 22076 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:45:19.142266 22076 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:45957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.33.84:36395/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:38143
--ranger_config_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:38143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=46809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:45:19.142576 22076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:45:19.142822 22076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:45:19.149307 22082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:19.149307 22084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:19.149349 22081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:45:19.149901 22076 server_base.cc:1034] running on GCE node
I20260928 14:45:19.150204 22076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:45:19.150555 22076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:45:19.151810 22076 hybrid_clock.cc:648] HybridClock initialized: now 1790606719151780 us; error 42 us; skew 500 ppm
Sep 28 14:45:19 dist-test-slave-pv0w krb5kdc[21108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606719, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:45:19.165103 22076 init.cc:377] Logged in from keytab as kudu/127.9.33.126@KRBTEST.COM (short username kudu)
I20260928 14:45:19.165791 22076 webserver.cc:460] Webserver started at http://127.9.33.126:46809/ using document root <none> and password file <none>
I20260928 14:45:19.166074 22076 fs_manager.cc:359] Metadata directory not provided
I20260928 14:45:19.166152 22076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 28 14:45:19 dist-test-slave-pv0w krb5kdc[21108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606719, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for HTTP/127.9.33.84@KRBTEST.COM
I20260928 14:45:19.221060 22076 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:45:19.221863 22092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:19.222041 22076 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:45:19.222164 22076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "84f7c09ac7624bcaa7e2c2512e47c346"
format_stamp: "Formatted at 2026-09-28 14:45:17 on dist-test-slave-pv0w"
server_key: "b17deaf4fcbe8ee31e9070ffb6f2cdc7"
server_key_iv: "af5a75a6c64c51fe20d2029c12ff0731"
server_key_version: "kuduclusterkey@0"
I20260928 14:45:19.222554 22076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:45:19.235702 22076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:45:19.237898 22076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:45:19.238068 22076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:45:19.242791 22076 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:38143
I20260928 14:45:19.242831 22144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:38143 every 8 connection(s)
I20260928 14:45:19.243227 22076 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:45:19.243897 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 22076
I20260928 14:45:19.244190 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.33.84:36395/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.65:37539
--local_ip_for_outbound_sockets=127.9.33.65
--tserver_master_addrs=127.9.33.126:38143
--webserver_port=34625
--webserver_interface=127.9.33.65
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:45957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.33.84:36395/kms with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:45:19.244664 22145 sys_catalog.cc:263] Verifying existing consensus state
I20260928 14:45:19.245365 22145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346: Bootstrap starting.
I20260928 14:45:19.253736 22145 log.cc:824] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346: Log is configured to *not* fsync() on all Append() calls
I20260928 14:45:19.255527 22145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 14:45:19.255885 22145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346: Bootstrap complete.
I20260928 14:45:19.258278 22145 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84f7c09ac7624bcaa7e2c2512e47c346" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38143 } }
I20260928 14:45:19.258548 22145 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84f7c09ac7624bcaa7e2c2512e47c346, State: Initialized, Role: FOLLOWER
I20260928 14:45:19.258742 22145 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84f7c09ac7624bcaa7e2c2512e47c346" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38143 } }
I20260928 14:45:19.258837 22145 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:45:19.258891 22145 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:45:19.258941 22145 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346 [term 1 FOLLOWER]: Advancing to term 2
I20260928 14:45:19.260036 22145 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84f7c09ac7624bcaa7e2c2512e47c346" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38143 } }
I20260928 14:45:19.260164 22145 leader_election.cc:302] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84f7c09ac7624bcaa7e2c2512e47c346; no voters:
I20260928 14:45:19.260344 22145 leader_election.cc:288] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260928 14:45:19.260414 22149 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346 [term 2 FOLLOWER]: Leader election won for term 2
I20260928 14:45:19.260560 22149 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346 [term 2 LEADER]: Becoming Leader. State: Replica: 84f7c09ac7624bcaa7e2c2512e47c346, State: Running, Role: LEADER
I20260928 14:45:19.260628 22145 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:45:19.260771 22149 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84f7c09ac7624bcaa7e2c2512e47c346" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38143 } }
I20260928 14:45:19.261587 22151 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84f7c09ac7624bcaa7e2c2512e47c346. Latest consensus state: current_term: 2 leader_uuid: "84f7c09ac7624bcaa7e2c2512e47c346" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84f7c09ac7624bcaa7e2c2512e47c346" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38143 } } }
I20260928 14:45:19.261677 22151 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346 [sys.catalog]: This master's current role is: LEADER
I20260928 14:45:19.261859 22159 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:45:19.262333 22150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "84f7c09ac7624bcaa7e2c2512e47c346" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84f7c09ac7624bcaa7e2c2512e47c346" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 38143 } } }
I20260928 14:45:19.262394 22159 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:45:19.262408 22150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346 [sys.catalog]: This master's current role is: LEADER
I20260928 14:45:19.262527 22159 catalog_manager.cc:1259] Loaded cluster ID: 062e8a252f4449089ba40ca2022a690a
I20260928 14:45:19.262579 22159 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:45:19.262800 22145 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260928 14:45:19.263558 22159 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:45:19.263701 22159 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346: Loaded TSK: 0
I20260928 14:45:19.263917 22159 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 14:45:19.359383 22147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:45:19.359668 22147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:45:19.359737 22147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:45:19.361768 22147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:45:19.361853 22147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:45:19.361898 22147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.65
I20260928 14:45:19.364670 22147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:45957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.33.84:36395/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.65:37539
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.33.65
--webserver_port=34625
--tserver_master_addrs=127.9.33.126:38143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.65
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:45:19.364984 22147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:45:19.365311 22147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:45:19.372670 22185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:19.372684 22182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:19.372676 22183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:45:19.373431 22147 server_base.cc:1034] running on GCE node
I20260928 14:45:19.373674 22147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:45:19.373992 22147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:45:19.375195 22147 hybrid_clock.cc:648] HybridClock initialized: now 1790606719375164 us; error 46 us; skew 500 ppm
Sep 28 14:45:19 dist-test-slave-pv0w krb5kdc[21108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606719, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:45:19.378801 22147 init.cc:377] Logged in from keytab as kudu/127.9.33.65@KRBTEST.COM (short username kudu)
I20260928 14:45:19.379418 22147 webserver.cc:460] Webserver started at http://127.9.33.65:34625/ using document root <none> and password file <none>
I20260928 14:45:19.379781 22147 fs_manager.cc:359] Metadata directory not provided
I20260928 14:45:19.379878 22147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 28 14:45:19 dist-test-slave-pv0w krb5kdc[21108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606719, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for HTTP/127.9.33.84@KRBTEST.COM
I20260928 14:45:19.404870 22147 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.000s sys 0.001s
I20260928 14:45:19.407938 22193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:19.408236 22147 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260928 14:45:19.408346 22147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-0/wal
uuid: "c1d06384fce2442aa53ee42518d4da95"
format_stamp: "Formatted at 2026-09-28 14:45:18 on dist-test-slave-pv0w"
server_key: "d61fa51ef5839e42328dad656a90197f"
server_key_iv: "71c5061ce1738d2ce785ba78e376fb2e"
server_key_version: "kuduclusterkey@0"
I20260928 14:45:19.408895 22147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:45:19.487084 22147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:45:19.489605 22147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:45:19.489792 22147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:45:19.490046 22147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:45:19.490453 22147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:45:19.490506 22147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:19.490549 22147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:45:19.490576 22147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:19.498124 22147 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.65:37539
I20260928 14:45:19.498596 22147 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
I20260928 14:45:19.503063 22306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.65:37539 every 8 connection(s)
I20260928 14:45:19.504559 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 22147
I20260928 14:45:19.504892 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.33.84:36395/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.66:41621
--local_ip_for_outbound_sockets=127.9.33.66
--tserver_master_addrs=127.9.33.126:38143
--webserver_port=41679
--webserver_interface=127.9.33.66
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:45957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.33.84:36395/kms with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 14:45:19 dist-test-slave-pv0w krb5kdc[21108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606719, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:45:19.522107 22309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:19.504053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38143 (local address 127.9.33.65:60725)
0928 14:45:19.504316 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:19.506611 (+ 2295us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:19.507366 (+ 755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:19.509299 (+ 1933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:19.509304 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:19.509611 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:45:19.509835 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:19.509840 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:19.511772 (+ 1932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:19.511774 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:19.513135 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:19.513139 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:19.514277 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:19.514279 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:19.514472 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:19.515222 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:45:19.515235 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:45:19.517062 (+ 1827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:45:19.519866 (+ 2804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:19.519875 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:19.519879 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:19.520258 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:19.520619 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:19.520621 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:19.520621 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:19.520800 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:19.521371 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:45:19.521378 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:45:19.521547 (+ 169us) client_negotiation.cc:769] Sending connection context
0928 14:45:19.521650 (+ 103us) client_negotiation.cc:240] Negotiation successful
0928 14:45:19.521840 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":106,"threads_started":1}
I20260928 14:45:19.523219 22310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:19.504126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:60725 (local address 127.9.33.126:38143)
0928 14:45:19.508662 (+ 4536us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:19.508675 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:19.508747 (+ 72us) server_negotiation.cc:407] Connection header received
0928 14:45:19.508942 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:19.508950 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:19.509025 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:19.509234 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:19.509965 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:19.511580 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:19.513270 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:19.514109 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:19.514842 (+ 733us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:19.517230 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:19.517236 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:19.517237 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:19.517254 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:19.519689 (+ 2435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:19.520384 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:19.520387 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:19.520388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:19.520522 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:19.520901 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:19.520902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:19.520903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:19.521274 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:19.522768 (+ 1494us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:19.522853 (+ 85us) server_negotiation.cc:299] Negotiation successful
0928 14:45:19.522999 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4448,"thread_start_us":117,"threads_started":1}
I20260928 14:45:19.524362 22307 heartbeater.cc:347] Connected to a master server at 127.9.33.126:38143
I20260928 14:45:19.524469 22307 heartbeater.cc:464] Registering TS with master...
I20260928 14:45:19.524673 22307 heartbeater.cc:511] Master 127.9.33.126:38143 requested a full tablet report, sending...
W20260928 14:45:19.525655 22307 heartbeater.cc:644] Failed to heartbeat to 127.9.33.126:38143 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260928 14:45:19.525743 22307 heartbeater.cc:464] Registering TS with master...
I20260928 14:45:19.525811 22307 heartbeater.cc:511] Master 127.9.33.126:38143 requested a full tablet report, sending...
I20260928 14:45:19.526202 22307 heartbeater.cc:464] Registering TS with master...
I20260928 14:45:19.526280 22307 heartbeater.cc:511] Master 127.9.33.126:38143 requested a full tablet report, sending...
W20260928 14:45:19.526664 22307 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 14:45:19.630005 22311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:45:19.630343 22311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:45:19.630393 22311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:45:19.632321 22311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:45:19.632390 22311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:45:19.632436 22311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.66
I20260928 14:45:19.634897 22311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:45957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.33.84:36395/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.66:41621
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.33.66
--webserver_port=41679
--tserver_master_addrs=127.9.33.126:38143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.66
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:45:19.635185 22311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:45:19.635509 22311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:45:19.642395 22317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:19.642395 22319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:19.642481 22316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:45:19.642755 22311 server_base.cc:1034] running on GCE node
I20260928 14:45:19.642973 22311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:45:19.643247 22311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:45:19.644444 22311 hybrid_clock.cc:648] HybridClock initialized: now 1790606719644418 us; error 46 us; skew 500 ppm
Sep 28 14:45:19 dist-test-slave-pv0w krb5kdc[21108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606719, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:45:19.647171 22311 init.cc:377] Logged in from keytab as kudu/127.9.33.66@KRBTEST.COM (short username kudu)
I20260928 14:45:19.647686 22311 webserver.cc:460] Webserver started at http://127.9.33.66:41679/ using document root <none> and password file <none>
I20260928 14:45:19.647961 22311 fs_manager.cc:359] Metadata directory not provided
I20260928 14:45:19.648042 22311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 28 14:45:19 dist-test-slave-pv0w krb5kdc[21108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606719, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for HTTP/127.9.33.84@KRBTEST.COM
I20260928 14:45:19.662732 22311 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260928 14:45:19.663815 22327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:19.663972 22311 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:45:19.664052 22311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-1/wal
uuid: "a7f71b99321a4396bf449f3adbab4ce2"
format_stamp: "Formatted at 2026-09-28 14:45:18 on dist-test-slave-pv0w"
server_key: "e5f1cefba0567a312f7c950b3997c5f7"
server_key_iv: "dcb45ebf36570fbe247d3feaa388873d"
server_key_version: "kuduclusterkey@0"
I20260928 14:45:19.664506 22311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:45:19.700243 22311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:45:19.702780 22311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:45:19.702998 22311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:45:19.703256 22311 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:45:19.703631 22311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:45:19.703684 22311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:19.703737 22311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:45:19.703764 22311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:19.710124 22311 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.66:41621
I20260928 14:45:19.710899 22311 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
I20260928 14:45:19.714594 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 22311
I20260928 14:45:19.714882 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.33.84:36395/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.67:36323
--local_ip_for_outbound_sockets=127.9.33.67
--tserver_master_addrs=127.9.33.126:38143
--webserver_port=36683
--webserver_interface=127.9.33.67
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:45957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.33.84:36395/kms with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:45:19.722330 22440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.66:41621 every 8 connection(s)
Sep 28 14:45:19 dist-test-slave-pv0w krb5kdc[21108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606719, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:45:19.734310 22310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:19.723030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:41125 (local address 127.9.33.126:38143)
0928 14:45:19.723100 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:19.723102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:19.723915 (+ 813us) server_negotiation.cc:407] Connection header received
0928 14:45:19.724636 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:19.724637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:19.724653 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:19.724683 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:19.725577 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:19.726417 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:19.727971 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:19.728280 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:19.728330 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:19.730966 (+ 2636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:19.730968 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:19.730968 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:19.730977 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:19.732764 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:19.733285 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:19.733286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:19.733287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:19.733351 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:19.733608 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:19.733609 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:19.733609 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:19.733732 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:19.733817 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:19.734111 (+ 294us) server_negotiation.cc:299] Negotiation successful
0928 14:45:19.734201 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260928 14:45:19.734725 22444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:19.722719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38143 (local address 127.9.33.66:41125)
0928 14:45:19.723757 (+ 1038us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:19.723816 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:19.724505 (+ 689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:19.724885 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:19.724890 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:19.725176 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:45:19.725467 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:19.725473 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:19.726552 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:19.726553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:19.727873 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:19.727876 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:19.728449 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:19.728450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:19.728595 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:19.729243 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:45:19.729254 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:45:19.730843 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:45:19.732881 (+ 2038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:19.732888 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:19.732891 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:19.733195 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:19.733430 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:19.733430 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:19.733431 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:19.733534 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:19.733824 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:45:19.733831 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:45:19.733969 (+ 138us) client_negotiation.cc:769] Sending connection context
0928 14:45:19.734079 (+ 110us) client_negotiation.cc:240] Negotiation successful
0928 14:45:19.734239 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":961,"thread_start_us":98,"threads_started":1}
I20260928 14:45:19.735430 22441 heartbeater.cc:347] Connected to a master server at 127.9.33.126:38143
I20260928 14:45:19.735528 22441 heartbeater.cc:464] Registering TS with master...
I20260928 14:45:19.735756 22441 heartbeater.cc:511] Master 127.9.33.126:38143 requested a full tablet report, sending...
W20260928 14:45:19.736441 22441 heartbeater.cc:644] Failed to heartbeat to 127.9.33.126:38143 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260928 14:45:19.736521 22441 heartbeater.cc:464] Registering TS with master...
I20260928 14:45:19.736590 22441 heartbeater.cc:511] Master 127.9.33.126:38143 requested a full tablet report, sending...
I20260928 14:45:19.736871 22441 heartbeater.cc:464] Registering TS with master...
I20260928 14:45:19.736944 22441 heartbeater.cc:511] Master 127.9.33.126:38143 requested a full tablet report, sending...
W20260928 14:45:19.737210 22441 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 14:45:19.839192 22443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:45:19.839536 22443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:45:19.839618 22443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:45:19.841563 22443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:45:19.841684 22443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:45:19.841751 22443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.67
I20260928 14:45:19.844225 22443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:45957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.33.84:36395/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.67:36323
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.33.67
--webserver_port=36683
--tserver_master_addrs=127.9.33.126:38143
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.67
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:45:19.844535 22443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:45:19.844806 22443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:45:19.850451 22453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:19.850466 22451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:19.850517 22450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:45:19.850827 22443 server_base.cc:1034] running on GCE node
I20260928 14:45:19.851104 22443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:45:19.851521 22443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:45:19.852690 22443 hybrid_clock.cc:648] HybridClock initialized: now 1790606719852664 us; error 51 us; skew 500 ppm
Sep 28 14:45:19 dist-test-slave-pv0w krb5kdc[21108](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606719, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:45:19.855818 22443 init.cc:377] Logged in from keytab as kudu/127.9.33.67@KRBTEST.COM (short username kudu)
I20260928 14:45:19.856383 22443 webserver.cc:460] Webserver started at http://127.9.33.67:36683/ using document root <none> and password file <none>
I20260928 14:45:19.856637 22443 fs_manager.cc:359] Metadata directory not provided
I20260928 14:45:19.856717 22443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 28 14:45:19 dist-test-slave-pv0w krb5kdc[21108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606719, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for HTTP/127.9.33.84@KRBTEST.COM
I20260928 14:45:19.877628 22443 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:45:19.878641 22461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:19.878834 22443 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:45:19.878932 22443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-2/wal
uuid: "07a94cc819234e2d888e84c1c5e3177a"
format_stamp: "Formatted at 2026-09-28 14:45:18 on dist-test-slave-pv0w"
server_key: "c00fa7309762fdd96f5f953bca4324cb"
server_key_iv: "4aaa2ba11688884e89f7cc4f3cbde150"
server_key_version: "kuduclusterkey@0"
I20260928 14:45:19.879350 22443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:45:19.898839 22443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:45:19.901247 22443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:45:19.901515 22443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:45:19.901815 22443 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:45:19.902212 22443 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:45:19.902251 22443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:19.902274 22443 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:45:19.902287 22443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:19.909976 22443 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.67:36323
I20260928 14:45:19.910197 22574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.67:36323 every 8 connection(s)
I20260928 14:45:19.910518 22443 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
I20260928 14:45:19.916313 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 22443
Sep 28 14:45:19 dist-test-slave-pv0w krb5kdc[21108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606719, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:45:19.921206 22577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:19.911329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:38143 (local address 127.9.33.67:33631)
0928 14:45:19.911509 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:19.911555 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:19.912134 (+ 579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:19.912384 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:19.912389 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:19.912590 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:45:19.912803 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:19.912808 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:19.913590 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:19.913592 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:19.914571 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:19.914574 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:19.915058 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:19.915060 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:19.915255 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:19.915883 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:45:19.915891 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:45:19.917444 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:45:19.919467 (+ 2023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:19.919478 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:19.919482 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:19.919823 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:19.920058 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:19.920059 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:19.920059 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:19.920214 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:19.920507 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:45:19.920514 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:45:19.920706 (+ 192us) client_negotiation.cc:769] Sending connection context
0928 14:45:19.920809 (+ 103us) client_negotiation.cc:240] Negotiation successful
0928 14:45:19.920993 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":103,"threads_started":1}
I20260928 14:45:19.921562 22310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:19.911337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:33631 (local address 127.9.33.126:38143)
0928 14:45:19.911412 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:19.911415 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:19.911637 (+ 222us) server_negotiation.cc:407] Connection header received
0928 14:45:19.912253 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:19.912255 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:19.912268 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:19.912310 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:19.912908 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:19.913492 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:19.914653 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:19.914948 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:19.915022 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:19.917554 (+ 2532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:19.917556 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:19.917556 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:19.917565 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:19.919357 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:19.919908 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:19.919910 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:19.919910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:19.919981 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:19.920290 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:19.920291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:19.920291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:19.920424 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:19.920482 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:19.921395 (+ 913us) server_negotiation.cc:299] Negotiation successful
0928 14:45:19.921489 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260928 14:45:19.922322 22575 heartbeater.cc:347] Connected to a master server at 127.9.33.126:38143
I20260928 14:45:19.922438 22575 heartbeater.cc:464] Registering TS with master...
I20260928 14:45:19.922669 22575 heartbeater.cc:511] Master 127.9.33.126:38143 requested a full tablet report, sending...
W20260928 14:45:19.923406 22575 heartbeater.cc:644] Failed to heartbeat to 127.9.33.126:38143 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260928 14:45:19.923486 22575 heartbeater.cc:464] Registering TS with master...
I20260928 14:45:19.923563 22575 heartbeater.cc:511] Master 127.9.33.126:38143 requested a full tablet report, sending...
I20260928 14:45:19.923939 22575 heartbeater.cc:464] Registering TS with master...
I20260928 14:45:19.924021 22575 heartbeater.cc:511] Master 127.9.33.126:38143 requested a full tablet report, sending...
W20260928 14:45:19.924347 22575 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260928 14:45:19.928921 22579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:19.916952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57260 (local address 127.9.33.126:38143)
0928 14:45:19.917273 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:19.917275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:19.917286 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:45:19.917615 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:19.917616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:19.917630 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:19.917725 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:19.918535 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:19.919370 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:19.921767 (+ 2397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:19.922081 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:19.922157 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:19.924742 (+ 2585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:19.924744 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:19.924744 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:19.924752 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:19.926470 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:19.927803 (+ 1333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:19.927804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:19.927805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:19.927880 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:19.928128 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:19.928130 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:19.928130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:19.928280 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:19.928461 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:19.928695 (+ 234us) server_negotiation.cc:299] Negotiation successful
0928 14:45:19.928807 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":49,"threads_started":1}
Sep 28 14:45:20 dist-test-slave-pv0w krb5kdc[21108](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790606720, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:45:20.526855 22307 heartbeater.cc:464] Registering TS with master...
I20260928 14:45:20.527030 22307 heartbeater.cc:511] Master 127.9.33.126:38143 requested a full tablet report, sending...
I20260928 14:45:20.527488 22109 ts_manager.cc:194] Registered new tserver with Master: c1d06384fce2442aa53ee42518d4da95 (127.9.33.65:37539)
I20260928 14:45:20.528533 22109 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.65@KRBTEST.COM'} at 127.9.33.65:60725
I20260928 14:45:20.737404 22441 heartbeater.cc:464] Registering TS with master...
I20260928 14:45:20.737568 22441 heartbeater.cc:511] Master 127.9.33.126:38143 requested a full tablet report, sending...
I20260928 14:45:20.737845 22108 ts_manager.cc:194] Registered new tserver with Master: a7f71b99321a4396bf449f3adbab4ce2 (127.9.33.66:41621)
I20260928 14:45:20.738369 22108 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.66@KRBTEST.COM'} at 127.9.33.66:41125
Sep 28 14:45:20 dist-test-slave-pv0w krb5kdc[21108](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606720, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for HTTP/127.9.33.84@KRBTEST.COM
I20260928 14:45:20.924562 22575 heartbeater.cc:464] Registering TS with master...
I20260928 14:45:20.924746 22575 heartbeater.cc:511] Master 127.9.33.126:38143 requested a full tablet report, sending...
I20260928 14:45:20.925172 22108 ts_manager.cc:194] Registered new tserver with Master: 07a94cc819234e2d888e84c1c5e3177a (127.9.33.67:36323)
I20260928 14:45:20.925698 22108 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.67@KRBTEST.COM'} at 127.9.33.67:33631
I20260928 14:45:20.925736 9349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:45:20.935495 22597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:20.927020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57274 (local address 127.9.33.126:38143)
0928 14:45:20.927156 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:20.927158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:20.927271 (+ 113us) server_negotiation.cc:407] Connection header received
0928 14:45:20.927326 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:20.927327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:20.927343 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:20.927433 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:20.928179 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:20.929030 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:20.930068 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:20.930410 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:20.930502 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:20.932059 (+ 1557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:20.932061 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:20.932061 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:20.932070 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:20.934063 (+ 1993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:20.934515 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:20.934516 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:20.934517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:20.934595 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:20.934828 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:20.934829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:20.934829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:20.934995 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:20.935108 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:20.935262 (+ 154us) server_negotiation.cc:299] Negotiation successful
0928 14:45:20.935369 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":65,"threads_started":1}
I20260928 14:45:20.938023 22108 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57274:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:45:20.940795 22108 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:45:20.950793 22603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:20.944561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:41621 (local address 127.0.0.1:46068)
0928 14:45:20.945373 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:20.945415 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:20.945490 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:20.945825 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:20.945828 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:20.945831 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:45:20.945921 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:20.945923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:20.947510 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:20.947511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:20.949178 (+ 1667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:20.949181 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:20.950397 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:20.950398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:20.950501 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:20.950519 (+ 18us) client_negotiation.cc:769] Sending connection context
0928 14:45:20.950603 (+ 84us) client_negotiation.cc:240] Negotiation successful
0928 14:45:20.950670 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":782,"thread_start_us":109,"threads_started":1}
I20260928 14:45:20.951182 22607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:20.945050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46068 (local address 127.9.33.66:41621)
0928 14:45:20.945194 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:20.945199 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:20.945567 (+ 368us) server_negotiation.cc:407] Connection header received
0928 14:45:20.945682 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:20.945695 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:20.945714 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:20.945814 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:45:20.946151 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:20.947391 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:20.949300 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:20.950185 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:20.950274 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:20.950352 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:20.951034 (+ 682us) server_negotiation.cc:299] Negotiation successful
0928 14:45:20.951090 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":90,"threads_started":1}
I20260928 14:45:20.952008 22375 tablet_service.cc:1467] Processing CreateTablet for tablet e93932b3d9fe42c2a7424e87487ad0a8 (DEFAULT_TABLE table=test-table [id=c1d1da70a461495aa18e0de783a6a9ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:45:20.952458 22375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e93932b3d9fe42c2a7424e87487ad0a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:45:20.956246 22602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:20.944565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:36323 (local address 127.0.0.1:33548)
0928 14:45:20.947849 (+ 3284us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:20.947861 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:20.947952 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:20.948209 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:20.948210 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:20.948212 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:45:20.948253 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:20.948255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:20.950132 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:20.950133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:20.954540 (+ 4407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:20.954543 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:20.955625 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:20.955627 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:20.955691 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:20.955695 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:45:20.956110 (+ 415us) client_negotiation.cc:240] Negotiation successful
0928 14:45:20.956163 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3264,"thread_start_us":105,"threads_started":1}
I20260928 14:45:20.956637 22605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:20.944896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33548 (local address 127.9.33.67:36323)
0928 14:45:20.945303 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:20.945307 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:20.947922 (+ 2615us) server_negotiation.cc:407] Connection header received
0928 14:45:20.948030 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:20.948033 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:20.948049 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:20.948102 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:45:20.948500 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:20.949840 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:20.954638 (+ 4798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:20.955447 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:20.955500 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:20.955556 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:20.956510 (+ 954us) server_negotiation.cc:299] Negotiation successful
0928 14:45:20.956565 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":61,"threads_started":1}
I20260928 14:45:20.957232 22608 tablet_bootstrap.cc:492] T e93932b3d9fe42c2a7424e87487ad0a8 P a7f71b99321a4396bf449f3adbab4ce2: Bootstrap starting.
I20260928 14:45:20.958042 22608 tablet_bootstrap.cc:654] T e93932b3d9fe42c2a7424e87487ad0a8 P a7f71b99321a4396bf449f3adbab4ce2: Neither blocks nor log segments found. Creating new log.
I20260928 14:45:20.958264 22606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:20.944953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34094 (local address 127.9.33.65:37539)
0928 14:45:20.945079 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:20.945085 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:20.948397 (+ 3312us) server_negotiation.cc:407] Connection header received
0928 14:45:20.949949 (+ 1552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:20.949953 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:20.949971 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:20.950024 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:45:20.951412 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:20.952992 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:20.956929 (+ 3937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:20.957676 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:20.957726 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:20.957794 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:20.958076 (+ 282us) server_negotiation.cc:299] Negotiation successful
0928 14:45:20.958183 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":66,"threads_started":1}
I20260928 14:45:20.958418 22608 log.cc:824] T e93932b3d9fe42c2a7424e87487ad0a8 P a7f71b99321a4396bf449f3adbab4ce2: Log is configured to *not* fsync() on all Append() calls
I20260928 14:45:20.958948 22604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:20.944846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:37539 (local address 127.0.0.1:34094)
0928 14:45:20.948332 (+ 3486us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:20.948342 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:20.948425 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:20.951153 (+ 2728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:20.951154 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:20.951156 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:45:20.951193 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:20.951195 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:20.953084 (+ 1889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:20.953084 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:20.954156 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:20.954160 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:20.957914 (+ 3754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:20.957915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:20.957984 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:20.957987 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:45:20.958816 (+ 829us) client_negotiation.cc:240] Negotiation successful
0928 14:45:20.958872 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3471,"thread_start_us":88,"threads_started":1}
I20260928 14:45:20.959354 22608 tablet_bootstrap.cc:492] T e93932b3d9fe42c2a7424e87487ad0a8 P a7f71b99321a4396bf449f3adbab4ce2: No bootstrap required, opened a new log
I20260928 14:45:20.959299 22508 tablet_service.cc:1467] Processing CreateTablet for tablet e93932b3d9fe42c2a7424e87487ad0a8 (DEFAULT_TABLE table=test-table [id=c1d1da70a461495aa18e0de783a6a9ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:45:20.959444 22608 ts_tablet_manager.cc:1397] T e93932b3d9fe42c2a7424e87487ad0a8 P a7f71b99321a4396bf449f3adbab4ce2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:45:20.959658 22508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e93932b3d9fe42c2a7424e87487ad0a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:45:20.962152 22608 raft_consensus.cc:348] T e93932b3d9fe42c2a7424e87487ad0a8 P a7f71b99321a4396bf449f3adbab4ce2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07a94cc819234e2d888e84c1c5e3177a" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 36323 } } peers { permanent_uuid: "a7f71b99321a4396bf449f3adbab4ce2" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 41621 } } peers { permanent_uuid: "c1d06384fce2442aa53ee42518d4da95" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37539 } }
I20260928 14:45:20.962358 22608 raft_consensus.cc:374] T e93932b3d9fe42c2a7424e87487ad0a8 P a7f71b99321a4396bf449f3adbab4ce2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:45:20.962404 22608 raft_consensus.cc:729] T e93932b3d9fe42c2a7424e87487ad0a8 P a7f71b99321a4396bf449f3adbab4ce2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7f71b99321a4396bf449f3adbab4ce2, State: Initialized, Role: FOLLOWER
I20260928 14:45:20.962813 22610 tablet_bootstrap.cc:492] T e93932b3d9fe42c2a7424e87487ad0a8 P 07a94cc819234e2d888e84c1c5e3177a: Bootstrap starting.
I20260928 14:45:20.963065 22241 tablet_service.cc:1467] Processing CreateTablet for tablet e93932b3d9fe42c2a7424e87487ad0a8 (DEFAULT_TABLE table=test-table [id=c1d1da70a461495aa18e0de783a6a9ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:45:20.963526 22241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e93932b3d9fe42c2a7424e87487ad0a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:45:20.963621 22610 tablet_bootstrap.cc:654] T e93932b3d9fe42c2a7424e87487ad0a8 P 07a94cc819234e2d888e84c1c5e3177a: Neither blocks nor log segments found. Creating new log.
I20260928 14:45:20.964013 22610 log.cc:824] T e93932b3d9fe42c2a7424e87487ad0a8 P 07a94cc819234e2d888e84c1c5e3177a: Log is configured to *not* fsync() on all Append() calls
I20260928 14:45:20.964780 22610 tablet_bootstrap.cc:492] T e93932b3d9fe42c2a7424e87487ad0a8 P 07a94cc819234e2d888e84c1c5e3177a: No bootstrap required, opened a new log
I20260928 14:45:20.964844 22610 ts_tablet_manager.cc:1397] T e93932b3d9fe42c2a7424e87487ad0a8 P 07a94cc819234e2d888e84c1c5e3177a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:45:20.967407 22610 raft_consensus.cc:348] T e93932b3d9fe42c2a7424e87487ad0a8 P 07a94cc819234e2d888e84c1c5e3177a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07a94cc819234e2d888e84c1c5e3177a" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 36323 } } peers { permanent_uuid: "a7f71b99321a4396bf449f3adbab4ce2" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 41621 } } peers { permanent_uuid: "c1d06384fce2442aa53ee42518d4da95" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37539 } }
I20260928 14:45:20.967592 22610 raft_consensus.cc:374] T e93932b3d9fe42c2a7424e87487ad0a8 P 07a94cc819234e2d888e84c1c5e3177a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:45:20.967626 22610 raft_consensus.cc:729] T e93932b3d9fe42c2a7424e87487ad0a8 P 07a94cc819234e2d888e84c1c5e3177a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07a94cc819234e2d888e84c1c5e3177a, State: Initialized, Role: FOLLOWER
I20260928 14:45:20.967708 22608 consensus_queue.cc:260] T e93932b3d9fe42c2a7424e87487ad0a8 P a7f71b99321a4396bf449f3adbab4ce2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07a94cc819234e2d888e84c1c5e3177a" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 36323 } } peers { permanent_uuid: "a7f71b99321a4396bf449f3adbab4ce2" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 41621 } } peers { permanent_uuid: "c1d06384fce2442aa53ee42518d4da95" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37539 } }
I20260928 14:45:20.968148 22608 ts_tablet_manager.cc:1428] T e93932b3d9fe42c2a7424e87487ad0a8 P a7f71b99321a4396bf449f3adbab4ce2: Time spent starting tablet: real 0.009s user 0.004s sys 0.000s
I20260928 14:45:20.968291 22441 heartbeater.cc:503] Master 127.9.33.126:38143 was elected leader, sending a full tablet report...
I20260928 14:45:20.968796 22610 consensus_queue.cc:260] T e93932b3d9fe42c2a7424e87487ad0a8 P 07a94cc819234e2d888e84c1c5e3177a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07a94cc819234e2d888e84c1c5e3177a" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 36323 } } peers { permanent_uuid: "a7f71b99321a4396bf449f3adbab4ce2" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 41621 } } peers { permanent_uuid: "c1d06384fce2442aa53ee42518d4da95" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37539 } }
I20260928 14:45:20.969161 22610 ts_tablet_manager.cc:1428] T e93932b3d9fe42c2a7424e87487ad0a8 P 07a94cc819234e2d888e84c1c5e3177a: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260928 14:45:20.969770 22575 heartbeater.cc:503] Master 127.9.33.126:38143 was elected leader, sending a full tablet report...
I20260928 14:45:20.969751 22612 tablet_bootstrap.cc:492] T e93932b3d9fe42c2a7424e87487ad0a8 P c1d06384fce2442aa53ee42518d4da95: Bootstrap starting.
I20260928 14:45:20.970520 22612 tablet_bootstrap.cc:654] T e93932b3d9fe42c2a7424e87487ad0a8 P c1d06384fce2442aa53ee42518d4da95: Neither blocks nor log segments found. Creating new log.
I20260928 14:45:20.970901 22612 log.cc:824] T e93932b3d9fe42c2a7424e87487ad0a8 P c1d06384fce2442aa53ee42518d4da95: Log is configured to *not* fsync() on all Append() calls
I20260928 14:45:20.972286 22612 tablet_bootstrap.cc:492] T e93932b3d9fe42c2a7424e87487ad0a8 P c1d06384fce2442aa53ee42518d4da95: No bootstrap required, opened a new log
I20260928 14:45:20.972376 22612 ts_tablet_manager.cc:1397] T e93932b3d9fe42c2a7424e87487ad0a8 P c1d06384fce2442aa53ee42518d4da95: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
W20260928 14:45:20.974453 22442 tablet.cc:2369] T e93932b3d9fe42c2a7424e87487ad0a8 P a7f71b99321a4396bf449f3adbab4ce2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:45:20.975404 22612 raft_consensus.cc:348] T e93932b3d9fe42c2a7424e87487ad0a8 P c1d06384fce2442aa53ee42518d4da95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07a94cc819234e2d888e84c1c5e3177a" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 36323 } } peers { permanent_uuid: "a7f71b99321a4396bf449f3adbab4ce2" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 41621 } } peers { permanent_uuid: "c1d06384fce2442aa53ee42518d4da95" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37539 } }
I20260928 14:45:20.975613 22612 raft_consensus.cc:374] T e93932b3d9fe42c2a7424e87487ad0a8 P c1d06384fce2442aa53ee42518d4da95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:45:20.975651 22612 raft_consensus.cc:729] T e93932b3d9fe42c2a7424e87487ad0a8 P c1d06384fce2442aa53ee42518d4da95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1d06384fce2442aa53ee42518d4da95, State: Initialized, Role: FOLLOWER
I20260928 14:45:20.975847 22612 consensus_queue.cc:260] T e93932b3d9fe42c2a7424e87487ad0a8 P c1d06384fce2442aa53ee42518d4da95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07a94cc819234e2d888e84c1c5e3177a" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 36323 } } peers { permanent_uuid: "a7f71b99321a4396bf449f3adbab4ce2" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 41621 } } peers { permanent_uuid: "c1d06384fce2442aa53ee42518d4da95" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37539 } }
I20260928 14:45:20.976164 22612 ts_tablet_manager.cc:1428] T e93932b3d9fe42c2a7424e87487ad0a8 P c1d06384fce2442aa53ee42518d4da95: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260928 14:45:20.976194 22307 heartbeater.cc:503] Master 127.9.33.126:38143 was elected leader, sending a full tablet report...
I20260928 14:45:20.983373 22614 raft_consensus.cc:482] T e93932b3d9fe42c2a7424e87487ad0a8 P 07a94cc819234e2d888e84c1c5e3177a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:45:20.983530 22614 raft_consensus.cc:504] T e93932b3d9fe42c2a7424e87487ad0a8 P 07a94cc819234e2d888e84c1c5e3177a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07a94cc819234e2d888e84c1c5e3177a" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 36323 } } peers { permanent_uuid: "a7f71b99321a4396bf449f3adbab4ce2" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 41621 } } peers { permanent_uuid: "c1d06384fce2442aa53ee42518d4da95" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37539 } }
I20260928 14:45:20.984007 22614 leader_election.cc:288] T e93932b3d9fe42c2a7424e87487ad0a8 P 07a94cc819234e2d888e84c1c5e3177a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a7f71b99321a4396bf449f3adbab4ce2 (127.9.33.66:41621), c1d06384fce2442aa53ee42518d4da95 (127.9.33.65:37539)
I20260928 14:45:20.988994 22618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:20.984262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:37539 (local address 127.9.33.67:46653)
0928 14:45:20.984816 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:20.984829 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:20.984892 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:20.985674 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:20.985675 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:20.985677 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:45:20.985781 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:20.985782 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:20.986641 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:20.986642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:20.987916 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:20.987919 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:20.988718 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:20.988719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:20.988798 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:20.988801 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:45:20.988842 (+ 41us) client_negotiation.cc:240] Negotiation successful
0928 14:45:20.988885 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"mutex_wait_us":2,"thread_start_us":117,"threads_started":1}
I20260928 14:45:20.989166 22607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:20.984728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:35703 (local address 127.9.33.66:41621)
0928 14:45:20.984785 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:20.984787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:20.985205 (+ 418us) server_negotiation.cc:407] Connection header received
0928 14:45:20.985322 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:20.985323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:20.985337 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:20.985369 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:45:20.985636 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:20.986432 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:20.988086 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:20.988715 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:20.988770 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:20.988782 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:20.989042 (+ 260us) server_negotiation.cc:299] Negotiation successful
0928 14:45:20.989089 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 14:45:20.989269 22606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:20.985017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:46653 (local address 127.9.33.65:37539)
0928 14:45:20.985400 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:20.985402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:20.985415 (+ 13us) server_negotiation.cc:407] Connection header received
0928 14:45:20.985444 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:20.985445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:20.985458 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:20.985528 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:45:20.985860 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:20.986543 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:20.988022 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:20.988555 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:20.988603 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:20.988614 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:20.989165 (+ 551us) server_negotiation.cc:299] Negotiation successful
0928 14:45:20.989208 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355}
I20260928 14:45:20.989588 22247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e93932b3d9fe42c2a7424e87487ad0a8" candidate_uuid: "07a94cc819234e2d888e84c1c5e3177a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1d06384fce2442aa53ee42518d4da95" is_pre_election: true
I20260928 14:45:20.989665 22617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:20.984519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:41621 (local address 127.9.33.67:35703)
0928 14:45:20.985138 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:20.985149 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:20.985237 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:20.985446 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:20.985447 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:20.985448 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:45:20.985544 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:20.985546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:20.986560 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:20.986561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:20.987961 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:20.987965 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:20.988852 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:20.988853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:20.988939 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:20.988943 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:45:20.989521 (+ 578us) client_negotiation.cc:240] Negotiation successful
0928 14:45:20.989572 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"mutex_wait_us":28,"thread_start_us":84,"threads_started":1}
I20260928 14:45:20.989806 22247 raft_consensus.cc:2454] T e93932b3d9fe42c2a7424e87487ad0a8 P c1d06384fce2442aa53ee42518d4da95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07a94cc819234e2d888e84c1c5e3177a in term 0.
I20260928 14:45:20.990017 22377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e93932b3d9fe42c2a7424e87487ad0a8" candidate_uuid: "07a94cc819234e2d888e84c1c5e3177a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7f71b99321a4396bf449f3adbab4ce2" is_pre_election: true
I20260928 14:45:20.990063 22464 leader_election.cc:302] T e93932b3d9fe42c2a7424e87487ad0a8 P 07a94cc819234e2d888e84c1c5e3177a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07a94cc819234e2d888e84c1c5e3177a, c1d06384fce2442aa53ee42518d4da95; no voters:
I20260928 14:45:20.990188 22377 raft_consensus.cc:2454] T e93932b3d9fe42c2a7424e87487ad0a8 P a7f71b99321a4396bf449f3adbab4ce2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07a94cc819234e2d888e84c1c5e3177a in term 0.
I20260928 14:45:20.990204 22614 raft_consensus.cc:2781] T e93932b3d9fe42c2a7424e87487ad0a8 P 07a94cc819234e2d888e84c1c5e3177a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:45:20.990252 22614 raft_consensus.cc:482] T e93932b3d9fe42c2a7424e87487ad0a8 P 07a94cc819234e2d888e84c1c5e3177a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:45:20.990275 22614 raft_consensus.cc:3037] T e93932b3d9fe42c2a7424e87487ad0a8 P 07a94cc819234e2d888e84c1c5e3177a [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:45:20.991186 22614 raft_consensus.cc:504] T e93932b3d9fe42c2a7424e87487ad0a8 P 07a94cc819234e2d888e84c1c5e3177a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07a94cc819234e2d888e84c1c5e3177a" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 36323 } } peers { permanent_uuid: "a7f71b99321a4396bf449f3adbab4ce2" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 41621 } } peers { permanent_uuid: "c1d06384fce2442aa53ee42518d4da95" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37539 } }
I20260928 14:45:20.991325 22614 leader_election.cc:288] T e93932b3d9fe42c2a7424e87487ad0a8 P 07a94cc819234e2d888e84c1c5e3177a [CANDIDATE]: Term 1 election: Requested vote from peers a7f71b99321a4396bf449f3adbab4ce2 (127.9.33.66:41621), c1d06384fce2442aa53ee42518d4da95 (127.9.33.65:37539)
I20260928 14:45:20.991554 22377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e93932b3d9fe42c2a7424e87487ad0a8" candidate_uuid: "07a94cc819234e2d888e84c1c5e3177a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7f71b99321a4396bf449f3adbab4ce2"
I20260928 14:45:20.991624 22377 raft_consensus.cc:3037] T e93932b3d9fe42c2a7424e87487ad0a8 P a7f71b99321a4396bf449f3adbab4ce2 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:45:20.992515 22377 raft_consensus.cc:2454] T e93932b3d9fe42c2a7424e87487ad0a8 P a7f71b99321a4396bf449f3adbab4ce2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07a94cc819234e2d888e84c1c5e3177a in term 1.
I20260928 14:45:20.992702 22463 leader_election.cc:302] T e93932b3d9fe42c2a7424e87487ad0a8 P 07a94cc819234e2d888e84c1c5e3177a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07a94cc819234e2d888e84c1c5e3177a, a7f71b99321a4396bf449f3adbab4ce2; no voters:
I20260928 14:45:20.992791 22614 raft_consensus.cc:2781] T e93932b3d9fe42c2a7424e87487ad0a8 P 07a94cc819234e2d888e84c1c5e3177a [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:45:20.992929 22614 raft_consensus.cc:686] T e93932b3d9fe42c2a7424e87487ad0a8 P 07a94cc819234e2d888e84c1c5e3177a [term 1 LEADER]: Becoming Leader. State: Replica: 07a94cc819234e2d888e84c1c5e3177a, State: Running, Role: LEADER
I20260928 14:45:20.993021 22614 consensus_queue.cc:237] T e93932b3d9fe42c2a7424e87487ad0a8 P 07a94cc819234e2d888e84c1c5e3177a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07a94cc819234e2d888e84c1c5e3177a" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 36323 } } peers { permanent_uuid: "a7f71b99321a4396bf449f3adbab4ce2" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 41621 } } peers { permanent_uuid: "c1d06384fce2442aa53ee42518d4da95" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37539 } }
I20260928 14:45:20.993788 22108 catalog_manager.cc:5495] T e93932b3d9fe42c2a7424e87487ad0a8 P 07a94cc819234e2d888e84c1c5e3177a reported cstate change: term changed from 0 to 1, leader changed from <none> to 07a94cc819234e2d888e84c1c5e3177a (127.9.33.67). New cstate: current_term: 1 leader_uuid: "07a94cc819234e2d888e84c1c5e3177a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07a94cc819234e2d888e84c1c5e3177a" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 36323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a7f71b99321a4396bf449f3adbab4ce2" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 41621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1d06384fce2442aa53ee42518d4da95" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37539 } health_report { overall_health: UNKNOWN } } }
I20260928 14:45:20.995988 22247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e93932b3d9fe42c2a7424e87487ad0a8" candidate_uuid: "07a94cc819234e2d888e84c1c5e3177a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1d06384fce2442aa53ee42518d4da95"
I20260928 14:45:20.996111 22247 raft_consensus.cc:3037] T e93932b3d9fe42c2a7424e87487ad0a8 P c1d06384fce2442aa53ee42518d4da95 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:45:20.997123 22247 raft_consensus.cc:2454] T e93932b3d9fe42c2a7424e87487ad0a8 P c1d06384fce2442aa53ee42518d4da95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07a94cc819234e2d888e84c1c5e3177a in term 1.
W20260928 14:45:21.004931 22308 tablet.cc:2369] T e93932b3d9fe42c2a7424e87487ad0a8 P c1d06384fce2442aa53ee42518d4da95: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:45:21.010810 22605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:21.007472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33556 (local address 127.9.33.67:36323)
0928 14:45:21.007533 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:21.007536 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:21.007582 (+ 46us) server_negotiation.cc:407] Connection header received
0928 14:45:21.007684 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:21.007685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:21.007739 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:21.007821 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:45:21.007954 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:21.008741 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:21.009804 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:21.010142 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:21.010245 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:21.010404 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:45:21.010559 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:45:21.010664 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:21.010693 (+ 29us) server_negotiation.cc:299] Negotiation successful
0928 14:45:21.010720 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260928 14:45:21.013132 22247 raft_consensus.cc:1261] T e93932b3d9fe42c2a7424e87487ad0a8 P c1d06384fce2442aa53ee42518d4da95 [term 1 FOLLOWER]: Refusing update from remote peer 07a94cc819234e2d888e84c1c5e3177a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:45:21.013170 22377 raft_consensus.cc:1261] T e93932b3d9fe42c2a7424e87487ad0a8 P a7f71b99321a4396bf449f3adbab4ce2 [term 1 FOLLOWER]: Refusing update from remote peer 07a94cc819234e2d888e84c1c5e3177a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:45:21.013556 22619 consensus_queue.cc:1035] T e93932b3d9fe42c2a7424e87487ad0a8 P 07a94cc819234e2d888e84c1c5e3177a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1d06384fce2442aa53ee42518d4da95" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 37539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:45:21.013675 22614 consensus_queue.cc:1035] T e93932b3d9fe42c2a7424e87487ad0a8 P 07a94cc819234e2d888e84c1c5e3177a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7f71b99321a4396bf449f3adbab4ce2" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 41621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:45:21.017611 22622 mvcc.cc:205] Tried to move back new op lower bound from 7334325129266962432 to 7334325129187946496. Current Snapshot: MvccSnapshot[applied={T|T < 7334325129266962432}]
I20260928 14:45:21.018349 22623 mvcc.cc:205] Tried to move back new op lower bound from 7334325129266962432 to 7334325129187946496. Current Snapshot: MvccSnapshot[applied={T|T < 7334325129266962432}]
I20260928 14:45:21.021047 22108 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57274:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:45:21.021168 22108 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57274:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:45:21.022168 22108 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 84f7c09ac7624bcaa7e2c2512e47c346: Sending DeleteTablet for 3 replicas of tablet e93932b3d9fe42c2a7424e87487ad0a8
I20260928 14:45:21.022701 22241 tablet_service.cc:1514] Processing DeleteTablet for tablet e93932b3d9fe42c2a7424e87487ad0a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:45:21 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:34094
I20260928 14:45:21.023111 22508 tablet_service.cc:1514] Processing DeleteTablet for tablet e93932b3d9fe42c2a7424e87487ad0a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:45:21 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:33548
I20260928 14:45:21.023344 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 22147
I20260928 14:45:21.023514 22375 tablet_service.cc:1514] Processing DeleteTablet for tablet e93932b3d9fe42c2a7424e87487ad0a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:45:21 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:46068
I20260928 14:45:21.027904 22633 tablet_replica.cc:330] T e93932b3d9fe42c2a7424e87487ad0a8 P a7f71b99321a4396bf449f3adbab4ce2: stopping tablet replica
I20260928 14:45:21.028082 22633 raft_consensus.cc:2229] T e93932b3d9fe42c2a7424e87487ad0a8 P a7f71b99321a4396bf449f3adbab4ce2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:45:21.028261 22633 raft_consensus.cc:2258] T e93932b3d9fe42c2a7424e87487ad0a8 P a7f71b99321a4396bf449f3adbab4ce2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:45:21.028865 22633 ts_tablet_manager.cc:1905] T e93932b3d9fe42c2a7424e87487ad0a8 P a7f71b99321a4396bf449f3adbab4ce2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:45:21.031828 22633 ts_tablet_manager.cc:1918] T e93932b3d9fe42c2a7424e87487ad0a8 P a7f71b99321a4396bf449f3adbab4ce2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:45:21.031980 22633 log.cc:1196] T e93932b3d9fe42c2a7424e87487ad0a8 P a7f71b99321a4396bf449f3adbab4ce2: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-1/wal/wals/e93932b3d9fe42c2a7424e87487ad0a8
I20260928 14:45:21.032333 22633 ts_tablet_manager.cc:1939] T e93932b3d9fe42c2a7424e87487ad0a8 P a7f71b99321a4396bf449f3adbab4ce2: Deleting consensus metadata
I20260928 14:45:21.032887 22094 catalog_manager.cc:4841] TS a7f71b99321a4396bf449f3adbab4ce2 (127.9.33.66:41621): tablet e93932b3d9fe42c2a7424e87487ad0a8 (table test-table [id=c1d1da70a461495aa18e0de783a6a9ae]) successfully deleted
I20260928 14:45:21.043840 22632 tablet_replica.cc:330] T e93932b3d9fe42c2a7424e87487ad0a8 P 07a94cc819234e2d888e84c1c5e3177a: stopping tablet replica
I20260928 14:45:21.044035 22632 raft_consensus.cc:2229] T e93932b3d9fe42c2a7424e87487ad0a8 P 07a94cc819234e2d888e84c1c5e3177a [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:45:21.044178 22632 raft_consensus.cc:2258] T e93932b3d9fe42c2a7424e87487ad0a8 P 07a94cc819234e2d888e84c1c5e3177a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260928 14:45:21.044564 22095 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.33.65:37539 (error 108)
I20260928 14:45:21.044869 22632 ts_tablet_manager.cc:1905] T e93932b3d9fe42c2a7424e87487ad0a8 P 07a94cc819234e2d888e84c1c5e3177a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:45:21.048007 22632 ts_tablet_manager.cc:1918] T e93932b3d9fe42c2a7424e87487ad0a8 P 07a94cc819234e2d888e84c1c5e3177a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:45:21.048116 22632 log.cc:1196] T e93932b3d9fe42c2a7424e87487ad0a8 P 07a94cc819234e2d888e84c1c5e3177a: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790606561545179-9349-0/minicluster-data/ts-2/wal/wals/e93932b3d9fe42c2a7424e87487ad0a8
I20260928 14:45:21.048448 22632 ts_tablet_manager.cc:1939] T e93932b3d9fe42c2a7424e87487ad0a8 P 07a94cc819234e2d888e84c1c5e3177a: Deleting consensus metadata
I20260928 14:45:21.048990 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 22311
I20260928 14:45:21.049000 22095 catalog_manager.cc:4841] TS 07a94cc819234e2d888e84c1c5e3177a (127.9.33.67:36323): tablet e93932b3d9fe42c2a7424e87487ad0a8 (table test-table [id=c1d1da70a461495aa18e0de783a6a9ae]) successfully deleted
I20260928 14:45:21.057850 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 22443
I20260928 14:45:21.060405 22603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:21.060230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:37539 (local address 127.0.0.1:34108)
0928 14:45:21.060296 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:21.060349 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.65:37539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260928 14:45:21.060571 22095 catalog_manager.cc:4584] TS c1d06384fce2442aa53ee42518d4da95 (127.9.33.65:37539): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e93932b3d9fe42c2a7424e87487ad0a8: Network error: Client connection negotiation failed: client connection to 127.9.33.65:37539: connect: Connection refused (error 111)
I20260928 14:45:21.065761 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 22076
I20260928 14:45:21.073722 9349 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260928 14:45:21.418907 9349 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260928 14:45:21.419037 9349 mini_ranger.cc:66] Stopping Ranger...
I20260928 14:45:21.764093 9349 mini_ranger.cc:68] Stopped Ranger
2026-09-28 14:45:21.764 UTC [21138] LOG: received smart shutdown request
2026-09-28 14:45:21.768 UTC [21138] LOG: background worker "logical replication launcher" (PID 21147) exited with exit code 1
2026-09-28 14:45:21.769 UTC [21142] LOG: shutting down
2026-09-28 14:45:21.965 UTC [21138] LOG: database system is shut down
2026-09-28T14:45:21Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50954 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:45:22 dist-test-slave-pv0w krb5kdc[22644](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:45:22 dist-test-slave-pv0w krb5kdc[22644](info): set up 2 sockets
Sep 28 14:45:22 dist-test-slave-pv0w krb5kdc[22644](info): commencing operation
krb5kdc: starting...
W20260928 14:45:23.140419 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:45:23 dist-test-slave-pv0w krb5kdc[22644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:45:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:45:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.33.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
I20260928 14:45:23.266949 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:35455
--webserver_interface=127.9.33.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:46077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:35455,127.9.33.125:39921,127.9.33.124:41111
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:45:23.355049 22657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:45:23.355360 22657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:45:23.355404 22657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:45:23.356675 22657 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:45:23.356750 22657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:45:23.356776 22657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:45:23.356804 22657 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:45:23.356834 22657 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:45:23.358429 22657 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:46077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:35455,127.9.33.125:39921,127.9.33.124:41111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:35455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:45:23.358733 22657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:45:23.358975 22657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:45:23.364398 22663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:23.364440 22665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:23.364406 22662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:45:23.364441 22657 server_base.cc:1034] running on GCE node
I20260928 14:45:23.364853 22657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:45:23.365156 22657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:45:23.366403 22657 hybrid_clock.cc:648] HybridClock initialized: now 1790606723366379 us; error 38 us; skew 500 ppm
Sep 28 14:45:23 dist-test-slave-pv0w krb5kdc[22644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606723, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:45:23.369655 22657 init.cc:377] Logged in from keytab as kudu/127.9.33.126@KRBTEST.COM (short username kudu)
I20260928 14:45:23.370208 22657 webserver.cc:460] Webserver started at http://127.9.33.126:33525/ using document root <none> and password file <none>
I20260928 14:45:23.370424 22657 fs_manager.cc:359] Metadata directory not provided
I20260928 14:45:23.370468 22657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:45:23.370568 22657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:45:23.371618 22657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "d085dbdabba244a3b2e114dc777df787"
format_stamp: "Formatted at 2026-09-28 14:45:23 on dist-test-slave-pv0w"
server_key: "c98a18121d29de7ee8b50c2222e2ccee"
server_key_iv: "e27676de63d7550b9ca0a4cc8cdd0787"
server_key_version: "encryptionkey@0"
I20260928 14:45:23.372010 22657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "d085dbdabba244a3b2e114dc777df787"
format_stamp: "Formatted at 2026-09-28 14:45:23 on dist-test-slave-pv0w"
server_key: "c98a18121d29de7ee8b50c2222e2ccee"
server_key_iv: "e27676de63d7550b9ca0a4cc8cdd0787"
server_key_version: "encryptionkey@0"
I20260928 14:45:23.373399 22657 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:45:23.374223 22672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:23.374539 22657 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:45:23.374631 22657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "d085dbdabba244a3b2e114dc777df787"
format_stamp: "Formatted at 2026-09-28 14:45:23 on dist-test-slave-pv0w"
server_key: "c98a18121d29de7ee8b50c2222e2ccee"
server_key_iv: "e27676de63d7550b9ca0a4cc8cdd0787"
server_key_version: "encryptionkey@0"
I20260928 14:45:23.374696 22657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:45:23.390208 22657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:45:23.392627 22657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:45:23.392753 22657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:45:23.397440 22657 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:35455
I20260928 14:45:23.397475 22724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:35455 every 8 connection(s)
I20260928 14:45:23.397850 22657 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:45:23.398800 22725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:45:23.400573 22725 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35455 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:45:23.402843 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 22657
I20260928 14:45:23.402931 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:45:23.403167 9349 external_mini_cluster.cc:1428] Setting key e3a032383703f454c29f260808c8e6c4
Sep 28 14:45:23 dist-test-slave-pv0w krb5kdc[22644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606723, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.33.125@KRBTEST.COM; defaulting to no policy
I20260928 14:45:23.411177 22727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.401313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:35455 (local address 127.0.0.1:46876)
0928 14:45:23.401511 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:23.401604 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:23.402130 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:23.402457 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:23.402460 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:23.402756 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:45:23.402964 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:23.402970 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.404103 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.404104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:23.404811 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:23.404813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.405357 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.405358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:23.405424 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:23.405955 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:45:23.405967 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:45:23.407793 (+ 1826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:45:23.409731 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:23.409738 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:23.409741 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:23.409942 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:23.410198 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:23.410199 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:23.410200 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:23.410287 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:23.410655 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:45:23.410661 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:45:23.410727 (+ 66us) client_negotiation.cc:769] Sending connection context
0928 14:45:23.410840 (+ 113us) client_negotiation.cc:240] Negotiation successful
0928 14:45:23.410988 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":69,"threads_started":1}
I20260928 14:45:23.411499 22728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.401473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46876 (local address 127.9.33.126:35455)
0928 14:45:23.401604 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:23.401614 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:23.401713 (+ 99us) server_negotiation.cc:407] Connection header received
0928 14:45:23.402310 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:23.402312 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:23.402337 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:23.402438 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:23.403072 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.404012 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.404913 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.405233 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.405388 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:23.407896 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:23.407903 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:23.407904 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:23.407912 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:23.409585 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:23.410038 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:23.410042 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:23.410042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:23.410127 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:23.410351 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:23.410352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:23.410352 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:23.410585 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:23.410686 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:23.411360 (+ 674us) server_negotiation.cc:299] Negotiation successful
0928 14:45:23.411432 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":44,"threads_started":1}
I20260928 14:45:23.412328 22725 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.33.125" port: 39921 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:45:23.412793 22727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.412609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.125:39921 (local address 127.0.0.1:60592)
0928 14:45:23.412704 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:23.412761 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.125:39921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260928 14:45:23.412958 22673 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.33.125:39921: connect: Connection refused (error 111)
I20260928 14:45:23.413549 22727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.413392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.125:39921 (local address 127.0.0.1:60604)
0928 14:45:23.413489 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:23.413523 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.125:39921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.125@KRBTEST.COM" created.
W20260928 14:45:23.413704 22725 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.33.125:39921: Network error: Client connection negotiation failed: client connection to 127.9.33.125:39921: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.125.keytab.
Entry for principal kudu/127.9.33.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.125.keytab.
I20260928 14:45:23.431874 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.125:39921
--webserver_interface=127.9.33.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.125.keytab
--principal=kudu/127.9.33.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:46077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:35455,127.9.33.125:39921,127.9.33.124:41111
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:45:23.462965 22725 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.33.125" port: 39921 } attempt: 1
I20260928 14:45:23.463590 22727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.463445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.125:39921 (local address 127.0.0.1:60610)
0928 14:45:23.463518 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:23.463558 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.125:39921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260928 14:45:23.464089 22727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.463955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.125:39921 (local address 127.0.0.1:60612)
0928 14:45:23.464028 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:23.464062 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.125:39921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260928 14:45:23.464319 22725 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.33.125:39921: Network error: Client connection negotiation failed: client connection to 127.9.33.125:39921: connect: Connection refused (error 111)
W20260928 14:45:23.516582 22732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:45:23.516810 22732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:45:23.516834 22732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:45:23.518047 22732 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:45:23.518095 22732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:45:23.518110 22732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:45:23.518122 22732 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:45:23.518157 22732 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:45:23.519793 22732 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:46077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:35455,127.9.33.125:39921,127.9.33.124:41111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.125.keytab
--principal=kudu/127.9.33.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.125:39921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.33.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:45:23.520063 22732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:45:23.520303 22732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:45:23.525300 22737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:23.525314 22738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:45:23.525370 22732 server_base.cc:1034] running on GCE node
W20260928 14:45:23.525329 22740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:45:23.525835 22732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:45:23.526144 22732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:45:23.527357 22732 hybrid_clock.cc:648] HybridClock initialized: now 1790606723527363 us; error 60 us; skew 500 ppm
Sep 28 14:45:23 dist-test-slave-pv0w krb5kdc[22644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606723, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:45:23.530165 22732 init.cc:377] Logged in from keytab as kudu/127.9.33.125@KRBTEST.COM (short username kudu)
I20260928 14:45:23.530731 22732 webserver.cc:460] Webserver started at http://127.9.33.125:42207/ using document root <none> and password file <none>
I20260928 14:45:23.530965 22732 fs_manager.cc:359] Metadata directory not provided
I20260928 14:45:23.531039 22732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:45:23.531193 22732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:45:23.532380 22732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-1/data/instance:
uuid: "c9b9d3a08bea4ef2a7faca26aa016fe1"
format_stamp: "Formatted at 2026-09-28 14:45:23 on dist-test-slave-pv0w"
server_key: "6e7ca2d67fda4ff7c09d711008411363"
server_key_iv: "4a8ccc11b7bf6b8fa836df73a5a496c9"
server_key_version: "encryptionkey@0"
I20260928 14:45:23.532469 22725 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.33.125" port: 39921 } attempt: 2
I20260928 14:45:23.532785 22732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-1/wal/instance:
uuid: "c9b9d3a08bea4ef2a7faca26aa016fe1"
format_stamp: "Formatted at 2026-09-28 14:45:23 on dist-test-slave-pv0w"
server_key: "6e7ca2d67fda4ff7c09d711008411363"
server_key_iv: "4a8ccc11b7bf6b8fa836df73a5a496c9"
server_key_version: "encryptionkey@0"
I20260928 14:45:23.532912 22727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.532765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.125:39921 (local address 127.0.0.1:60624)
0928 14:45:23.532823 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:23.532865 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.125:39921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260928 14:45:23.533288 22727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.533173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.125:39921 (local address 127.0.0.1:60626)
0928 14:45:23.533224 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:23.533257 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.125:39921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260928 14:45:23.533439 22725 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.33.125:39921: Network error: Client connection negotiation failed: client connection to 127.9.33.125:39921: connect: Connection refused (error 111)
I20260928 14:45:23.534724 22732 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260928 14:45:23.535440 22747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:23.535614 22732 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 14:45:23.535766 22732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-1/wal
uuid: "c9b9d3a08bea4ef2a7faca26aa016fe1"
format_stamp: "Formatted at 2026-09-28 14:45:23 on dist-test-slave-pv0w"
server_key: "6e7ca2d67fda4ff7c09d711008411363"
server_key_iv: "4a8ccc11b7bf6b8fa836df73a5a496c9"
server_key_version: "encryptionkey@0"
I20260928 14:45:23.535847 22732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:45:23.555161 22732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:45:23.557622 22732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:45:23.557755 22732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:45:23.561818 22732 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.125:39921
I20260928 14:45:23.561869 22799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.125:39921 every 8 connection(s)
I20260928 14:45:23.562232 22732 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-1/data/info.pb
I20260928 14:45:23.563184 22800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:45:23.564846 22800 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35455 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:45:23.567364 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 22732
I20260928 14:45:23.567476 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-1/wal/instance
I20260928 14:45:23.567636 9349 external_mini_cluster.cc:1428] Setting key 445688fc55f065ddeab75b3a226b3949
Sep 28 14:45:23 dist-test-slave-pv0w krb5kdc[22644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606723, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.125@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.33.124@KRBTEST.COM; defaulting to no policy
I20260928 14:45:23.575064 22728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.565662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46878 (local address 127.9.33.126:35455)
0928 14:45:23.565795 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:23.565798 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:23.565901 (+ 103us) server_negotiation.cc:407] Connection header received
0928 14:45:23.566439 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:23.566440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:23.566452 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:23.566500 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:23.567263 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.568066 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.569360 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.569572 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.569654 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:23.572270 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:23.572272 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:23.572272 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:23.572280 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:23.573653 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:23.574094 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:23.574095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:23.574096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:23.574158 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:23.574422 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:23.574423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:23.574423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:23.574539 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:23.574612 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:23.574883 (+ 271us) server_negotiation.cc:299] Negotiation successful
0928 14:45:23.574970 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260928 14:45:23.575273 22802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.565589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:35455 (local address 127.0.0.1:46878)
0928 14:45:23.565774 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:23.565817 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:23.566351 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:23.566568 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:23.566571 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:23.566834 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:45:23.567073 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:23.567078 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.568182 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.568183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:23.569258 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:23.569262 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.569667 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.569668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:23.569832 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:23.570454 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:45:23.570465 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:45:23.572162 (+ 1697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:45:23.573775 (+ 1613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:23.573782 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:23.573785 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:23.574020 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:23.574231 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:23.574232 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:23.574233 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:23.574339 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:23.574604 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:45:23.574611 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:45:23.574757 (+ 146us) client_negotiation.cc:769] Sending connection context
0928 14:45:23.574864 (+ 107us) client_negotiation.cc:240] Negotiation successful
0928 14:45:23.575027 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":76,"threads_started":1}
I20260928 14:45:23.575911 22800 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.33.125" port: 39921 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.124@KRBTEST.COM" created.
Sep 28 14:45:23 dist-test-slave-pv0w krb5kdc[22644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606723, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.125@KRBTEST.COM for kudu/127.9.33.125@KRBTEST.COM
I20260928 14:45:23.584148 22802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.576177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.125:39921 (local address 127.0.0.1:60628)
0928 14:45:23.576240 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:23.576250 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:23.576291 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:23.576562 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:23.576563 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:23.576721 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:45:23.576766 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:23.576767 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.577765 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.577766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:23.578460 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:23.578463 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.579000 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.579001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:23.579093 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:23.579671 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:45:23.579676 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:45:23.580850 (+ 1174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:45:23.583036 (+ 2186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:23.583037 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:23.583038 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:23.583213 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:23.583461 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:23.583463 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:23.583463 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:23.583525 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:23.583890 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:45:23.583890 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:45:23.583931 (+ 41us) client_negotiation.cc:769] Sending connection context
0928 14:45:23.583987 (+ 56us) client_negotiation.cc:240] Negotiation successful
0928 14:45:23.584076 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260928 14:45:23.584406 22804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.576244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60628 (local address 127.9.33.125:39921)
0928 14:45:23.576422 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:23.576426 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:23.576442 (+ 16us) server_negotiation.cc:407] Connection header received
0928 14:45:23.576459 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:23.576461 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:23.576471 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:23.576548 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:23.576846 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.577648 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.578543 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.578902 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.578989 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:23.580929 (+ 1940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:23.580939 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:23.580940 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:23.580947 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:23.582934 (+ 1987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:23.583294 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:23.583298 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:23.583299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:23.583368 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:23.583598 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:23.583599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:23.583599 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:23.583825 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:23.583914 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:23.584030 (+ 116us) server_negotiation.cc:299] Negotiation successful
0928 14:45:23.584143 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":153,"thread_start_us":111,"threads_started":1}
I20260928 14:45:23.585012 22800 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.33.124" port: 41111 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:45:23.585414 22802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.585295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:41111 (local address 127.0.0.1:53108)
0928 14:45:23.585342 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:23.585394 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.124:41111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260928 14:45:23.585536 22750 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.33.124:41111: connect: Connection refused (error 111)
I20260928 14:45:23.585953 22802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.585861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:41111 (local address 127.0.0.1:53116)
0928 14:45:23.585905 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:23.585930 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.124:41111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260928 14:45:23.586058 22800 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.33.124:41111: Network error: Client connection negotiation failed: client connection to 127.9.33.124:41111: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.124.keytab.
Entry for principal kudu/127.9.33.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.124.keytab.
I20260928 14:45:23.595319 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.124:41111
--webserver_interface=127.9.33.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.124.keytab
--principal=kudu/127.9.33.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:46077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:35455,127.9.33.125:39921,127.9.33.124:41111
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:45:23.624609 22725 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.33.125" port: 39921 } attempt: 3
Sep 28 14:45:23 dist-test-slave-pv0w krb5kdc[22644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606723, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for kudu/127.9.33.125@KRBTEST.COM
I20260928 14:45:23.632011 22727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.625103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.125:39921 (local address 127.0.0.1:60636)
0928 14:45:23.625177 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:23.625200 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:23.625270 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:23.625423 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:23.625424 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:23.625594 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:45:23.625665 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:23.625666 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.626364 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.626366 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:23.626899 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:23.626901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.627305 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.627306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:23.627391 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:23.627902 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:45:23.627907 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:45:23.629180 (+ 1273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:45:23.630883 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:23.630884 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:23.630885 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:23.631089 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:23.631309 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:23.631310 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:23.631311 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:23.631376 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:23.631627 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:45:23.631628 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:45:23.631677 (+ 49us) client_negotiation.cc:769] Sending connection context
0928 14:45:23.631790 (+ 113us) client_negotiation.cc:240] Negotiation successful
0928 14:45:23.631901 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260928 14:45:23.632006 22804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.625194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60636 (local address 127.9.33.125:39921)
0928 14:45:23.625265 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:23.625267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:23.625281 (+ 14us) server_negotiation.cc:407] Connection header received
0928 14:45:23.625336 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:23.625337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:23.625350 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:23.625416 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:23.625748 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.626270 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.626988 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.627216 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.627304 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:23.629268 (+ 1964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:23.629270 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:23.629270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:23.629278 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:23.630759 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:23.631165 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:23.631167 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:23.631167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:23.631236 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:23.631439 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:23.631440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:23.631440 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:23.631561 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:23.631636 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:23.631815 (+ 179us) server_negotiation.cc:299] Negotiation successful
0928 14:45:23.631903 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260928 14:45:23.632619 22725 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.33.124" port: 41111 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:45:23.633011 22727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.632911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:41111 (local address 127.0.0.1:53118)
0928 14:45:23.632960 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:23.632990 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.124:41111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260928 14:45:23.633308 22727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.633223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:41111 (local address 127.0.0.1:53128)
0928 14:45:23.633263 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:23.633286 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.124:41111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260928 14:45:23.633396 22725 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.33.124:41111: Network error: Client connection negotiation failed: client connection to 127.9.33.124:41111: connect: Connection refused (error 111)
I20260928 14:45:23.635299 22800 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.33.124" port: 41111 } attempt: 1
I20260928 14:45:23.635704 22802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.635576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:41111 (local address 127.0.0.1:53134)
0928 14:45:23.635640 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:23.635670 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.124:41111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260928 14:45:23.636147 22802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.636042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:41111 (local address 127.0.0.1:53148)
0928 14:45:23.636086 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:23.636111 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.124:41111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260928 14:45:23.636228 22800 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.33.124:41111: Network error: Client connection negotiation failed: client connection to 127.9.33.124:41111: connect: Connection refused (error 111)
I20260928 14:45:23.664566 22725 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.33.124" port: 41111 } attempt: 1
I20260928 14:45:23.665081 22727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.664937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:41111 (local address 127.0.0.1:53154)
0928 14:45:23.665006 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:23.665049 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.124:41111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260928 14:45:23.665513 22727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.665408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:41111 (local address 127.0.0.1:53168)
0928 14:45:23.665457 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:23.665492 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.124:41111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260928 14:45:23.665609 22725 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.33.124:41111: Network error: Client connection negotiation failed: client connection to 127.9.33.124:41111: connect: Connection refused (error 111)
W20260928 14:45:23.680923 22807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:45:23.681147 22807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:45:23.681170 22807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:45:23.682504 22807 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:45:23.682552 22807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:45:23.682566 22807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:45:23.682579 22807 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:45:23.682592 22807 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:45:23.684221 22807 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:46077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:35455,127.9.33.125:39921,127.9.33.124:41111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.124.keytab
--principal=kudu/127.9.33.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.124:41111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.33.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22807
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:45:23.684495 22807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:45:23.684743 22807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:45:23.689998 22815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:23.689900 22813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:23.689903 22812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:45:23.689934 22807 server_base.cc:1034] running on GCE node
I20260928 14:45:23.690371 22807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:45:23.690629 22807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:45:23.691797 22807 hybrid_clock.cc:648] HybridClock initialized: now 1790606723691784 us; error 32 us; skew 500 ppm
Sep 28 14:45:23 dist-test-slave-pv0w krb5kdc[22644](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606723, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:45:23.694429 22807 init.cc:377] Logged in from keytab as kudu/127.9.33.124@KRBTEST.COM (short username kudu)
I20260928 14:45:23.694901 22807 webserver.cc:460] Webserver started at http://127.9.33.124:43775/ using document root <none> and password file <none>
I20260928 14:45:23.695150 22807 fs_manager.cc:359] Metadata directory not provided
I20260928 14:45:23.695225 22807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:45:23.695381 22807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:45:23.696559 22807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-2/data/instance:
uuid: "519b37074f5f41f58c6c222cc576541f"
format_stamp: "Formatted at 2026-09-28 14:45:23 on dist-test-slave-pv0w"
server_key: "8cc9a9e10debaafb537d975fc3c87011"
server_key_iv: "718c9a0e6b8a3c80e82926a7cbc2c3f7"
server_key_version: "encryptionkey@0"
I20260928 14:45:23.696920 22807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-2/wal/instance:
uuid: "519b37074f5f41f58c6c222cc576541f"
format_stamp: "Formatted at 2026-09-28 14:45:23 on dist-test-slave-pv0w"
server_key: "8cc9a9e10debaafb537d975fc3c87011"
server_key_iv: "718c9a0e6b8a3c80e82926a7cbc2c3f7"
server_key_version: "encryptionkey@0"
I20260928 14:45:23.698321 22807 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:45:23.699095 22822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:23.699302 22807 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:45:23.699404 22807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-2/wal
uuid: "519b37074f5f41f58c6c222cc576541f"
format_stamp: "Formatted at 2026-09-28 14:45:23 on dist-test-slave-pv0w"
server_key: "8cc9a9e10debaafb537d975fc3c87011"
server_key_iv: "718c9a0e6b8a3c80e82926a7cbc2c3f7"
server_key_version: "encryptionkey@0"
I20260928 14:45:23.699476 22807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:45:23.704423 22800 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.33.124" port: 41111 } attempt: 2
I20260928 14:45:23.704957 22802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.704815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:41111 (local address 127.0.0.1:53184)
0928 14:45:23.704876 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:23.704918 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.124:41111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260928 14:45:23.705291 22802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.705199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:41111 (local address 127.0.0.1:53188)
0928 14:45:23.705240 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:23.705266 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.124:41111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260928 14:45:23.705385 22800 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.33.124:41111: Network error: Client connection negotiation failed: client connection to 127.9.33.124:41111: connect: Connection refused (error 111)
I20260928 14:45:23.731684 22807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:45:23.734100 22807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:45:23.734279 22807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:45:23.738670 22807 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.124:41111
I20260928 14:45:23.738751 22874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.124:41111 every 8 connection(s)
I20260928 14:45:23.739120 22807 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-2/data/info.pb
I20260928 14:45:23.740156 22875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:45:23.740762 22725 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.33.124" port: 41111 } attempt: 2
I20260928 14:45:23.740886 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 22807
I20260928 14:45:23.740995 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790606561545179-9349-0/minicluster-data/master-2/wal/instance
I20260928 14:45:23.741194 9349 external_mini_cluster.cc:1428] Setting key a6e383cb27c180d17957bd75e9e25a3b
I20260928 14:45:23.743156 22875 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35455 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 14:45:23 dist-test-slave-pv0w krb5kdc[22644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606723, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for kudu/127.9.33.124@KRBTEST.COM
Sep 28 14:45:23 dist-test-slave-pv0w krb5kdc[22644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.124@KRBTEST.COM
Sep 28 14:45:23 dist-test-slave-pv0w krb5kdc[22644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606723, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.124@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:45:23.751878 22727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.741123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:41111 (local address 127.0.0.1:53190)
0928 14:45:23.741180 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:23.741210 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:23.741284 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:23.742467 (+ 1183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:23.742468 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:23.742682 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:45:23.742754 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:23.742757 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.744389 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.744390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:23.745175 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:23.745177 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.745858 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.745860 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:23.745925 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:23.746528 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:45:23.746534 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:45:23.748015 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:45:23.750247 (+ 2232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:23.750248 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:23.750249 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:23.750478 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:23.750735 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:23.750736 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:23.750737 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:23.750798 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:23.751357 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:45:23.751358 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:45:23.751419 (+ 61us) client_negotiation.cc:769] Sending connection context
0928 14:45:23.751582 (+ 163us) client_negotiation.cc:240] Negotiation successful
0928 14:45:23.751747 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":26}
I20260928 14:45:23.752096 22877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.741351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53190 (local address 127.9.33.124:41111)
0928 14:45:23.741564 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:23.741575 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:23.741611 (+ 36us) server_negotiation.cc:407] Connection header received
0928 14:45:23.742197 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:23.742207 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:23.742282 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:23.742837 (+ 555us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:23.742898 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.744228 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.745268 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.745774 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.745912 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:23.748121 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:23.748130 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:23.748130 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:23.748143 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:23.750169 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:23.750554 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:23.750557 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:23.750557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:23.750665 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:23.750911 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:23.750912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:23.750913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:23.751265 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:23.751356 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:23.751666 (+ 310us) server_negotiation.cc:299] Negotiation successful
0928 14:45:23.751854 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":243,"server-negotiator.queue_time_us":157,"thread_start_us":99,"threads_started":1}
I20260928 14:45:23.753095 22728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.743965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46894 (local address 127.9.33.126:35455)
0928 14:45:23.744015 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:23.744017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:23.744203 (+ 186us) server_negotiation.cc:407] Connection header received
0928 14:45:23.744359 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:23.744360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:23.744371 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:23.744409 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:23.744921 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.745552 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.746562 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.746777 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.746849 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:23.749680 (+ 2831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:23.749682 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:23.749682 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:23.749690 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:23.751761 (+ 2071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:23.752287 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:23.752288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:23.752288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:23.752357 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:23.752539 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:23.752540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:23.752541 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:23.752669 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:23.752751 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:23.752925 (+ 174us) server_negotiation.cc:299] Negotiation successful
0928 14:45:23.753008 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260928 14:45:23.753213 22880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.743889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:35455 (local address 127.0.0.1:46894)
0928 14:45:23.744081 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:23.744124 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:23.744262 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:23.744484 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:23.744486 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:23.744760 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:45:23.744821 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:23.744823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.745643 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.745644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:23.746464 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:23.746467 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.746862 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.746863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:23.746940 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:23.747439 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:45:23.747450 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:45:23.749592 (+ 2142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:45:23.751852 (+ 2260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:23.751857 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:23.751859 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:23.752133 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:23.752429 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:23.752429 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:23.752430 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:23.752490 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:23.752744 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:45:23.752752 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:45:23.752801 (+ 49us) client_negotiation.cc:769] Sending connection context
0928 14:45:23.752898 (+ 97us) client_negotiation.cc:240] Negotiation successful
0928 14:45:23.753037 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":78,"threads_started":1}
I20260928 14:45:23.753648 22875 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.33.125" port: 39921 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:45:23.753842 22879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.741797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53206 (local address 127.9.33.124:41111)
0928 14:45:23.741918 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:23.741920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:23.741926 (+ 6us) server_negotiation.cc:407] Connection header received
0928 14:45:23.742199 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:23.742244 (+ 45us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:23.742281 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:23.742433 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:23.743024 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.744169 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.745280 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.745776 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.746131 (+ 355us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:23.749660 (+ 3529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:23.749662 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:23.749662 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:23.749668 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:23.752181 (+ 2513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:23.752560 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:23.752561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:23.752561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:23.752619 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:23.752866 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:23.752867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:23.752867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:23.753024 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:23.753580 (+ 556us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:23.753621 (+ 41us) server_negotiation.cc:299] Negotiation successful
0928 14:45:23.753701 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":74,"threads_started":1}
I20260928 14:45:23.754603 22725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d085dbdabba244a3b2e114dc777df787: Bootstrap starting.
I20260928 14:45:23.755376 22725 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d085dbdabba244a3b2e114dc777df787: Neither blocks nor log segments found. Creating new log.
I20260928 14:45:23.755693 22725 log.cc:824] T 00000000000000000000000000000000 P d085dbdabba244a3b2e114dc777df787: Log is configured to *not* fsync() on all Append() calls
I20260928 14:45:23.756673 22725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d085dbdabba244a3b2e114dc777df787: No bootstrap required, opened a new log
Sep 28 14:45:23 dist-test-slave-pv0w krb5kdc[22644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606723, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.124@KRBTEST.COM for kudu/127.9.33.125@KRBTEST.COM
Sep 28 14:45:23 dist-test-slave-pv0w krb5kdc[22644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:45:23.759299 22725 raft_consensus.cc:348] T 00000000000000000000000000000000 P d085dbdabba244a3b2e114dc777df787 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d085dbdabba244a3b2e114dc777df787" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35455 } } peers { permanent_uuid: "c9b9d3a08bea4ef2a7faca26aa016fe1" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 39921 } } peers { permanent_uuid: "519b37074f5f41f58c6c222cc576541f" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 41111 } }
I20260928 14:45:23.759475 22725 raft_consensus.cc:374] T 00000000000000000000000000000000 P d085dbdabba244a3b2e114dc777df787 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:45:23.759555 22725 raft_consensus.cc:729] T 00000000000000000000000000000000 P d085dbdabba244a3b2e114dc777df787 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d085dbdabba244a3b2e114dc777df787, State: Initialized, Role: FOLLOWER
I20260928 14:45:23.759707 22725 consensus_queue.cc:260] T 00000000000000000000000000000000 P d085dbdabba244a3b2e114dc777df787 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d085dbdabba244a3b2e114dc777df787" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35455 } } peers { permanent_uuid: "c9b9d3a08bea4ef2a7faca26aa016fe1" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 39921 } } peers { permanent_uuid: "519b37074f5f41f58c6c222cc576541f" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 41111 } }
I20260928 14:45:23.760154 22725 sys_catalog.cc:564] T 00000000000000000000000000000000 P d085dbdabba244a3b2e114dc777df787 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:45:23.760272 22882 sys_catalog.cc:455] T 00000000000000000000000000000000 P d085dbdabba244a3b2e114dc777df787 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d085dbdabba244a3b2e114dc777df787" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35455 } } peers { permanent_uuid: "c9b9d3a08bea4ef2a7faca26aa016fe1" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 39921 } } peers { permanent_uuid: "519b37074f5f41f58c6c222cc576541f" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 41111 } } }
I20260928 14:45:23.760455 22882 sys_catalog.cc:458] T 00000000000000000000000000000000 P d085dbdabba244a3b2e114dc777df787 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:45:23.761844 22880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.753946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.125:39921 (local address 127.0.0.1:60644)
0928 14:45:23.754116 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:23.754126 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:23.754173 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:23.754363 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:23.754365 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:23.754507 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:45:23.754563 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:23.754565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.755477 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.755478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:23.756234 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:23.756236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.756685 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.756686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:23.756755 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:23.757160 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:45:23.757165 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:45:23.758765 (+ 1600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:45:23.760720 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:23.760721 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:23.760722 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:23.760959 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:23.761168 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:23.761169 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:23.761169 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:23.761227 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:23.761519 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:45:23.761520 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:45:23.761570 (+ 50us) client_negotiation.cc:769] Sending connection context
0928 14:45:23.761648 (+ 78us) client_negotiation.cc:240] Negotiation successful
0928 14:45:23.761745 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157}
I20260928 14:45:23.761844 22804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.754050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60644 (local address 127.9.33.125:39921)
0928 14:45:23.754090 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:23.754092 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:23.754169 (+ 77us) server_negotiation.cc:407] Connection header received
0928 14:45:23.754236 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:23.754237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:23.754249 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:23.754308 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:23.754655 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.755373 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.756329 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.756612 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.756696 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:23.758861 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:23.758863 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:23.758864 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:23.758872 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:23.760626 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:23.761043 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:23.761044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:23.761044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:23.761101 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:23.761315 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:23.761316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:23.761316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:23.761464 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:23.761528 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:23.761665 (+ 137us) server_negotiation.cc:299] Negotiation successful
0928 14:45:23.761754 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 14:45:23.762614 22875 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.33.124" port: 41111 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:45:23.763003 22728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.754448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46908 (local address 127.9.33.126:35455)
0928 14:45:23.754481 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:23.754483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:23.754524 (+ 41us) server_negotiation.cc:407] Connection header received
0928 14:45:23.754590 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:23.754591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:23.754600 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:23.754654 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:23.755125 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.755577 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.756508 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.756788 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.756863 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:23.759579 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:23.759580 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:23.759581 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:23.759588 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:23.761444 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:23.761984 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:23.761986 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:23.761986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:23.762062 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:23.762295 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:23.762296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:23.762296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:23.762424 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:23.762561 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:23.762812 (+ 251us) server_negotiation.cc:299] Negotiation successful
0928 14:45:23.762895 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260928 14:45:23.763033 22657 master_runner.cc:186] Error getting master registration for 127.9.33.125:39921: OK, instance_id { permanent_uuid: "c9b9d3a08bea4ef2a7faca26aa016fe1" instance_seqno: 1790606723560630 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 14:45:23.766228 22657 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260928 14:45:23.762622 22893 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d085dbdabba244a3b2e114dc777df787: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:45:23.766423 22893 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Sep 28 14:45:23 dist-test-slave-pv0w krb5kdc[22644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606723, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.124@KRBTEST.COM for kudu/127.9.33.124@KRBTEST.COM
Sep 28 14:45:23 dist-test-slave-pv0w krb5kdc[22644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.125@KRBTEST.COM
I20260928 14:45:23.771636 22880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.762854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:41111 (local address 127.0.0.1:53222)
0928 14:45:23.762940 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:23.762951 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:23.762998 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:23.763192 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:23.763193 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:23.763349 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:45:23.763392 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:23.763394 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.764400 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.764408 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:23.765390 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:23.765393 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.765930 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.765930 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:23.765985 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:23.766502 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:45:23.766508 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:45:23.768093 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:45:23.770193 (+ 2100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:23.770194 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:23.770195 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:23.770453 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:23.770704 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:23.770705 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:23.770706 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:23.770772 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:23.771173 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:45:23.771174 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:45:23.771299 (+ 125us) client_negotiation.cc:769] Sending connection context
0928 14:45:23.771397 (+ 98us) client_negotiation.cc:240] Negotiation successful
0928 14:45:23.771519 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260928 14:45:23.771772 22879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.762908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53222 (local address 127.9.33.124:41111)
0928 14:45:23.762947 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:23.762949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:23.762985 (+ 36us) server_negotiation.cc:407] Connection header received
0928 14:45:23.763058 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:23.763058 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:23.763070 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:23.763454 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:23.763473 (+ 19us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.764264 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.765498 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.765825 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.767069 (+ 1244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:23.768197 (+ 1128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:23.768198 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:23.768198 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:23.768205 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:23.770089 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:23.770539 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:23.770540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:23.770540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:23.770613 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:23.770863 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:23.770864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:23.770864 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:23.771106 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:23.771183 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:23.771432 (+ 249us) server_negotiation.cc:299] Negotiation successful
0928 14:45:23.771600 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":20}
I20260928 14:45:23.773823 22875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 519b37074f5f41f58c6c222cc576541f: Bootstrap starting.
I20260928 14:45:23.774266 22804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.763795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60650 (local address 127.9.33.125:39921)
0928 14:45:23.763830 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:23.763831 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:23.763839 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:45:23.763907 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:23.763908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:23.763917 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:23.763970 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:23.764686 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.765597 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.766685 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.766977 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.767050 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:23.770070 (+ 3020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:23.770071 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:23.770071 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:23.770079 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:23.772501 (+ 2422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:23.772970 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:23.772972 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:23.772973 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:23.773036 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:23.773260 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:23.773261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:23.773262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:23.773419 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:23.774017 (+ 598us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:23.774062 (+ 45us) server_negotiation.cc:299] Negotiation successful
0928 14:45:23.774158 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 14:45:23.774647 22875 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 519b37074f5f41f58c6c222cc576541f: Neither blocks nor log segments found. Creating new log.
I20260928 14:45:23.775012 22875 log.cc:824] T 00000000000000000000000000000000 P 519b37074f5f41f58c6c222cc576541f: Log is configured to *not* fsync() on all Append() calls
I20260928 14:45:23.775949 9349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:45:23.776036 22875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 519b37074f5f41f58c6c222cc576541f: No bootstrap required, opened a new log
I20260928 14:45:23.778304 22875 raft_consensus.cc:348] T 00000000000000000000000000000000 P 519b37074f5f41f58c6c222cc576541f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d085dbdabba244a3b2e114dc777df787" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35455 } } peers { permanent_uuid: "c9b9d3a08bea4ef2a7faca26aa016fe1" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 39921 } } peers { permanent_uuid: "519b37074f5f41f58c6c222cc576541f" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 41111 } }
I20260928 14:45:23.778475 22875 raft_consensus.cc:374] T 00000000000000000000000000000000 P 519b37074f5f41f58c6c222cc576541f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:45:23.778524 22875 raft_consensus.cc:729] T 00000000000000000000000000000000 P 519b37074f5f41f58c6c222cc576541f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 519b37074f5f41f58c6c222cc576541f, State: Initialized, Role: FOLLOWER
I20260928 14:45:23.778684 22875 consensus_queue.cc:260] T 00000000000000000000000000000000 P 519b37074f5f41f58c6c222cc576541f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d085dbdabba244a3b2e114dc777df787" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35455 } } peers { permanent_uuid: "c9b9d3a08bea4ef2a7faca26aa016fe1" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 39921 } } peers { permanent_uuid: "519b37074f5f41f58c6c222cc576541f" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 41111 } }
I20260928 14:45:23.779006 22903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 519b37074f5f41f58c6c222cc576541f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d085dbdabba244a3b2e114dc777df787" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35455 } } peers { permanent_uuid: "c9b9d3a08bea4ef2a7faca26aa016fe1" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 39921 } } peers { permanent_uuid: "519b37074f5f41f58c6c222cc576541f" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 41111 } } }
I20260928 14:45:23.779069 22875 sys_catalog.cc:564] T 00000000000000000000000000000000 P 519b37074f5f41f58c6c222cc576541f [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:45:23.780809 22903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 519b37074f5f41f58c6c222cc576541f [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:45:23.782660 22807 master_runner.cc:186] Error getting master registration for 127.9.33.125:39921: OK, instance_id { permanent_uuid: "c9b9d3a08bea4ef2a7faca26aa016fe1" instance_seqno: 1790606723560630 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 14:45:23.785305 22807 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 14:45:23.786449 22804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.777590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60666 (local address 127.9.33.125:39921)
0928 14:45:23.777642 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:23.777644 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:23.778187 (+ 543us) server_negotiation.cc:407] Connection header received
0928 14:45:23.778301 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:23.778303 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:23.778313 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:23.778355 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:23.779284 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.780119 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.781185 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.781462 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.781506 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:23.783302 (+ 1796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:23.783304 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:23.783304 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:23.783312 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:23.785118 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:23.785590 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:23.785591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:23.785592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:23.785650 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:23.785849 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:23.785850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:23.785851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:23.785980 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:23.786051 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:23.786269 (+ 218us) server_negotiation.cc:299] Negotiation successful
0928 14:45:23.786352 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
W20260928 14:45:23.782056 22914 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 519b37074f5f41f58c6c222cc576541f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:45:23.786757 22914 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:45:23.791949 22728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.777806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46914 (local address 127.9.33.126:35455)
0928 14:45:23.777848 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:23.777850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:23.779269 (+ 1419us) server_negotiation.cc:407] Connection header received
0928 14:45:23.779398 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:23.779399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:23.779411 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:23.779482 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:23.780559 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.781363 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.783091 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.783417 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.783462 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:23.786605 (+ 3143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:23.786607 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:23.786608 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:23.786617 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:23.789316 (+ 2699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:23.790819 (+ 1503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:23.790821 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:23.790822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:23.790916 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:23.791158 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:23.791159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:23.791160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:23.791327 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:23.791413 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:23.791657 (+ 244us) server_negotiation.cc:299] Negotiation successful
0928 14:45:23.791781 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 14:45:23.796542 22800 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.33.124" port: 41111 } attempt: 3
I20260928 14:45:23.797160 22879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.777982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53226 (local address 127.9.33.124:41111)
0928 14:45:23.778032 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:23.778035 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:23.785751 (+ 7716us) server_negotiation.cc:407] Connection header received
0928 14:45:23.785871 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:23.785873 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:23.785885 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:23.785919 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:23.787615 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.788488 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.791883 (+ 3395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.792208 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.792256 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:23.793881 (+ 1625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:23.793883 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:23.793884 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:23.793892 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:23.795687 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:23.796147 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:23.796148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:23.796149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:23.796217 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:23.796435 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:23.796436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:23.796437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:23.796593 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:23.796711 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:23.796964 (+ 253us) server_negotiation.cc:299] Negotiation successful
0928 14:45:23.797060 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
Sep 28 14:45:23 dist-test-slave-pv0w krb5kdc[22644](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606723, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.125@KRBTEST.COM for kudu/127.9.33.124@KRBTEST.COM
I20260928 14:45:23.805892 22802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.796856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:41111 (local address 127.0.0.1:53232)
0928 14:45:23.797006 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:23.797017 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:23.797056 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:23.797284 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:23.797286 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:23.797457 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:45:23.797509 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:23.797511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.798446 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.798447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:23.799403 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:23.799406 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.800324 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.800325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:23.800389 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:23.800897 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:45:23.800903 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:45:23.802594 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:45:23.804659 (+ 2065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:23.804661 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:23.804661 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:23.804905 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:23.805129 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:23.805130 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:23.805130 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:23.805197 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:23.805540 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:45:23.805541 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:45:23.805592 (+ 51us) client_negotiation.cc:769] Sending connection context
0928 14:45:23.805667 (+ 75us) client_negotiation.cc:240] Negotiation successful
0928 14:45:23.805773 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129}
I20260928 14:45:23.805917 22877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:23.797126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53232 (local address 127.9.33.124:41111)
0928 14:45:23.797163 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:23.797165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:23.797171 (+ 6us) server_negotiation.cc:407] Connection header received
0928 14:45:23.797190 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:23.797191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:23.797200 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:23.797243 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:23.797588 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.798355 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.799498 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:23.800203 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:23.800268 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:23.802692 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:23.802694 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:23.802695 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:23.802702 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:23.804559 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:23.804986 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:23.804987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:23.804988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:23.805055 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:23.805266 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:23.805267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:23.805267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:23.805470 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:23.805543 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:23.805705 (+ 162us) server_negotiation.cc:299] Negotiation successful
0928 14:45:23.805798 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 14:45:23.808758 22800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9b9d3a08bea4ef2a7faca26aa016fe1: Bootstrap starting.
I20260928 14:45:23.809577 22800 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9b9d3a08bea4ef2a7faca26aa016fe1: Neither blocks nor log segments found. Creating new log.
I20260928 14:45:23.809917 22800 log.cc:824] T 00000000000000000000000000000000 P c9b9d3a08bea4ef2a7faca26aa016fe1: Log is configured to *not* fsync() on all Append() calls
I20260928 14:45:23.811252 22800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9b9d3a08bea4ef2a7faca26aa016fe1: No bootstrap required, opened a new log
I20260928 14:45:23.813715 22800 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9b9d3a08bea4ef2a7faca26aa016fe1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d085dbdabba244a3b2e114dc777df787" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35455 } } peers { permanent_uuid: "c9b9d3a08bea4ef2a7faca26aa016fe1" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 39921 } } peers { permanent_uuid: "519b37074f5f41f58c6c222cc576541f" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 41111 } }
I20260928 14:45:23.813993 22800 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9b9d3a08bea4ef2a7faca26aa016fe1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:45:23.814035 22800 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9b9d3a08bea4ef2a7faca26aa016fe1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9b9d3a08bea4ef2a7faca26aa016fe1, State: Initialized, Role: FOLLOWER
I20260928 14:45:23.814266 22800 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9b9d3a08bea4ef2a7faca26aa016fe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d085dbdabba244a3b2e114dc777df787" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35455 } } peers { permanent_uuid: "c9b9d3a08bea4ef2a7faca26aa016fe1" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 39921 } } peers { permanent_uuid: "519b37074f5f41f58c6c222cc576541f" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 41111 } }
I20260928 14:45:23.814636 22800 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9b9d3a08bea4ef2a7faca26aa016fe1 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:45:23.819229 22732 master_runner.cc:263] No leader master found from master c9b9d3a08bea4ef2a7faca26aa016fe1
I20260928 14:45:23.819312 22732 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260928 14:45:23.820499 22928 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c9b9d3a08bea4ef2a7faca26aa016fe1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:45:23.820565 22928 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:45:23.820655 22917 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9b9d3a08bea4ef2a7faca26aa016fe1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d085dbdabba244a3b2e114dc777df787" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35455 } } peers { permanent_uuid: "c9b9d3a08bea4ef2a7faca26aa016fe1" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 39921 } } peers { permanent_uuid: "519b37074f5f41f58c6c222cc576541f" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 41111 } } }
I20260928 14:45:23.820806 22917 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9b9d3a08bea4ef2a7faca26aa016fe1 [sys.catalog]: This master's current role is: FOLLOWER
W20260928 14:45:23.893620 22678 tablet.cc:2369] T 00000000000000000000000000000000 P d085dbdabba244a3b2e114dc777df787: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:45:23.985016 22828 tablet.cc:2369] T 00000000000000000000000000000000 P 519b37074f5f41f58c6c222cc576541f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:45:23.998332 22917 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9b9d3a08bea4ef2a7faca26aa016fe1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:45:23.998514 22917 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9b9d3a08bea4ef2a7faca26aa016fe1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d085dbdabba244a3b2e114dc777df787" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35455 } } peers { permanent_uuid: "c9b9d3a08bea4ef2a7faca26aa016fe1" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 39921 } } peers { permanent_uuid: "519b37074f5f41f58c6c222cc576541f" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 41111 } }
I20260928 14:45:23.998817 22917 leader_election.cc:288] T 00000000000000000000000000000000 P c9b9d3a08bea4ef2a7faca26aa016fe1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d085dbdabba244a3b2e114dc777df787 (127.9.33.126:35455), 519b37074f5f41f58c6c222cc576541f (127.9.33.124:41111)
I20260928 14:45:23.999179 22699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c9b9d3a08bea4ef2a7faca26aa016fe1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d085dbdabba244a3b2e114dc777df787" is_pre_election: true
I20260928 14:45:23.999210 22849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c9b9d3a08bea4ef2a7faca26aa016fe1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "519b37074f5f41f58c6c222cc576541f" is_pre_election: true
I20260928 14:45:23.999364 22699 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d085dbdabba244a3b2e114dc777df787 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9b9d3a08bea4ef2a7faca26aa016fe1 in term 0.
I20260928 14:45:23.999389 22849 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 519b37074f5f41f58c6c222cc576541f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9b9d3a08bea4ef2a7faca26aa016fe1 in term 0.
I20260928 14:45:23.999583 22750 leader_election.cc:302] T 00000000000000000000000000000000 P c9b9d3a08bea4ef2a7faca26aa016fe1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c9b9d3a08bea4ef2a7faca26aa016fe1, d085dbdabba244a3b2e114dc777df787; no voters:
I20260928 14:45:23.999881 22917 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9b9d3a08bea4ef2a7faca26aa016fe1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:45:23.999954 22917 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9b9d3a08bea4ef2a7faca26aa016fe1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:45:23.999972 22917 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9b9d3a08bea4ef2a7faca26aa016fe1 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:45:24.000746 22917 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9b9d3a08bea4ef2a7faca26aa016fe1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d085dbdabba244a3b2e114dc777df787" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35455 } } peers { permanent_uuid: "c9b9d3a08bea4ef2a7faca26aa016fe1" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 39921 } } peers { permanent_uuid: "519b37074f5f41f58c6c222cc576541f" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 41111 } }
I20260928 14:45:24.000891 22917 leader_election.cc:288] T 00000000000000000000000000000000 P c9b9d3a08bea4ef2a7faca26aa016fe1 [CANDIDATE]: Term 1 election: Requested vote from peers d085dbdabba244a3b2e114dc777df787 (127.9.33.126:35455), 519b37074f5f41f58c6c222cc576541f (127.9.33.124:41111)
I20260928 14:45:24.001103 22849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c9b9d3a08bea4ef2a7faca26aa016fe1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "519b37074f5f41f58c6c222cc576541f"
I20260928 14:45:24.001114 22699 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c9b9d3a08bea4ef2a7faca26aa016fe1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d085dbdabba244a3b2e114dc777df787"
I20260928 14:45:24.001205 22699 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d085dbdabba244a3b2e114dc777df787 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:45:24.001216 22849 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 519b37074f5f41f58c6c222cc576541f [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:45:24.002044 22849 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 519b37074f5f41f58c6c222cc576541f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9b9d3a08bea4ef2a7faca26aa016fe1 in term 1.
I20260928 14:45:24.002128 22699 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d085dbdabba244a3b2e114dc777df787 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9b9d3a08bea4ef2a7faca26aa016fe1 in term 1.
I20260928 14:45:24.002301 22750 leader_election.cc:302] T 00000000000000000000000000000000 P c9b9d3a08bea4ef2a7faca26aa016fe1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 519b37074f5f41f58c6c222cc576541f, c9b9d3a08bea4ef2a7faca26aa016fe1; no voters:
I20260928 14:45:24.002456 22917 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9b9d3a08bea4ef2a7faca26aa016fe1 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:45:24.002647 22917 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9b9d3a08bea4ef2a7faca26aa016fe1 [term 1 LEADER]: Becoming Leader. State: Replica: c9b9d3a08bea4ef2a7faca26aa016fe1, State: Running, Role: LEADER
I20260928 14:45:24.002770 22917 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9b9d3a08bea4ef2a7faca26aa016fe1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d085dbdabba244a3b2e114dc777df787" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35455 } } peers { permanent_uuid: "c9b9d3a08bea4ef2a7faca26aa016fe1" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 39921 } } peers { permanent_uuid: "519b37074f5f41f58c6c222cc576541f" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 41111 } }
I20260928 14:45:24.003274 22930 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9b9d3a08bea4ef2a7faca26aa016fe1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9b9d3a08bea4ef2a7faca26aa016fe1. Latest consensus state: current_term: 1 leader_uuid: "c9b9d3a08bea4ef2a7faca26aa016fe1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d085dbdabba244a3b2e114dc777df787" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35455 } } peers { permanent_uuid: "c9b9d3a08bea4ef2a7faca26aa016fe1" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 39921 } } peers { permanent_uuid: "519b37074f5f41f58c6c222cc576541f" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 41111 } } }
I20260928 14:45:24.003412 22930 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9b9d3a08bea4ef2a7faca26aa016fe1 [sys.catalog]: This master's current role is: LEADER
I20260928 14:45:24.003641 22932 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:45:24.003870 22932 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:45:24.005376 22699 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d085dbdabba244a3b2e114dc777df787 [term 1 FOLLOWER]: Refusing update from remote peer c9b9d3a08bea4ef2a7faca26aa016fe1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:45:24.005416 22849 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 519b37074f5f41f58c6c222cc576541f [term 1 FOLLOWER]: Refusing update from remote peer c9b9d3a08bea4ef2a7faca26aa016fe1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:45:24.005787 22930 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c9b9d3a08bea4ef2a7faca26aa016fe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "519b37074f5f41f58c6c222cc576541f" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 41111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:45:24.005934 22917 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c9b9d3a08bea4ef2a7faca26aa016fe1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d085dbdabba244a3b2e114dc777df787" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:45:24.007227 22882 sys_catalog.cc:455] T 00000000000000000000000000000000 P d085dbdabba244a3b2e114dc777df787 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9b9d3a08bea4ef2a7faca26aa016fe1. Latest consensus state: current_term: 1 leader_uuid: "c9b9d3a08bea4ef2a7faca26aa016fe1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d085dbdabba244a3b2e114dc777df787" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35455 } } peers { permanent_uuid: "c9b9d3a08bea4ef2a7faca26aa016fe1" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 39921 } } peers { permanent_uuid: "519b37074f5f41f58c6c222cc576541f" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 41111 } } }
I20260928 14:45:24.007228 22903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 519b37074f5f41f58c6c222cc576541f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9b9d3a08bea4ef2a7faca26aa016fe1. Latest consensus state: current_term: 1 leader_uuid: "c9b9d3a08bea4ef2a7faca26aa016fe1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d085dbdabba244a3b2e114dc777df787" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35455 } } peers { permanent_uuid: "c9b9d3a08bea4ef2a7faca26aa016fe1" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 39921 } } peers { permanent_uuid: "519b37074f5f41f58c6c222cc576541f" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 41111 } } }
I20260928 14:45:24.007345 22903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 519b37074f5f41f58c6c222cc576541f [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:45:24.007347 22882 sys_catalog.cc:458] T 00000000000000000000000000000000 P d085dbdabba244a3b2e114dc777df787 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:45:24.008430 22917 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9b9d3a08bea4ef2a7faca26aa016fe1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c9b9d3a08bea4ef2a7faca26aa016fe1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d085dbdabba244a3b2e114dc777df787" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35455 } } peers { permanent_uuid: "c9b9d3a08bea4ef2a7faca26aa016fe1" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 39921 } } peers { permanent_uuid: "519b37074f5f41f58c6c222cc576541f" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 41111 } } }
I20260928 14:45:24.008533 22917 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9b9d3a08bea4ef2a7faca26aa016fe1 [sys.catalog]: This master's current role is: LEADER
I20260928 14:45:24.008586 22917 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9b9d3a08bea4ef2a7faca26aa016fe1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c9b9d3a08bea4ef2a7faca26aa016fe1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d085dbdabba244a3b2e114dc777df787" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35455 } } peers { permanent_uuid: "c9b9d3a08bea4ef2a7faca26aa016fe1" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 39921 } } peers { permanent_uuid: "519b37074f5f41f58c6c222cc576541f" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 41111 } } }
I20260928 14:45:24.008627 22917 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9b9d3a08bea4ef2a7faca26aa016fe1 [sys.catalog]: This master's current role is: LEADER
I20260928 14:45:24.008594 22935 mvcc.cc:205] Tried to move back new op lower bound from 7334325141523333120 to 7334325141516042240. Current Snapshot: MvccSnapshot[applied={T|T < 7334325141523333120}]
I20260928 14:45:24.008692 22882 sys_catalog.cc:455] T 00000000000000000000000000000000 P d085dbdabba244a3b2e114dc777df787 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c9b9d3a08bea4ef2a7faca26aa016fe1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d085dbdabba244a3b2e114dc777df787" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35455 } } peers { permanent_uuid: "c9b9d3a08bea4ef2a7faca26aa016fe1" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 39921 } } peers { permanent_uuid: "519b37074f5f41f58c6c222cc576541f" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 41111 } } }
I20260928 14:45:24.008776 22882 sys_catalog.cc:458] T 00000000000000000000000000000000 P d085dbdabba244a3b2e114dc777df787 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:45:24.008747 22903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 519b37074f5f41f58c6c222cc576541f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c9b9d3a08bea4ef2a7faca26aa016fe1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d085dbdabba244a3b2e114dc777df787" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35455 } } peers { permanent_uuid: "c9b9d3a08bea4ef2a7faca26aa016fe1" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 39921 } } peers { permanent_uuid: "519b37074f5f41f58c6c222cc576541f" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 41111 } } }
I20260928 14:45:24.008816 22903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 519b37074f5f41f58c6c222cc576541f [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:45:24.009735 22932 catalog_manager.cc:1347] Generated new cluster ID: 93f1c9cc6a054e16909c1d5eb58a0dea
I20260928 14:45:24.009795 22932 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:45:24.035815 22932 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:45:24.036559 22932 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:45:24.041671 22932 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9b9d3a08bea4ef2a7faca26aa016fe1: Generated new TSK 0
I20260928 14:45:24.041905 22932 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:45:24.767016 22893 catalog_manager.cc:1259] Loaded cluster ID: 93f1c9cc6a054e16909c1d5eb58a0dea
I20260928 14:45:24.767140 22893 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d085dbdabba244a3b2e114dc777df787: loading cluster ID for follower catalog manager: success
I20260928 14:45:24.768052 22893 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d085dbdabba244a3b2e114dc777df787: acquiring CA information for follower catalog manager: success
I20260928 14:45:24.768401 22893 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d085dbdabba244a3b2e114dc777df787: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 14:45:24.787271 22914 catalog_manager.cc:1259] Loaded cluster ID: 93f1c9cc6a054e16909c1d5eb58a0dea
I20260928 14:45:24.787351 22914 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 519b37074f5f41f58c6c222cc576541f: loading cluster ID for follower catalog manager: success
I20260928 14:45:24.788306 22914 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 519b37074f5f41f58c6c222cc576541f: acquiring CA information for follower catalog manager: success
I20260928 14:45:24.788641 22914 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 519b37074f5f41f58c6c222cc576541f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 14:45:25.122867 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 22657
I20260928 14:45:25.128162 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 22732
I20260928 14:45:25.133008 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 22807
2026-09-28T14:45:25Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3057 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:45:25 dist-test-slave-pv0w krb5kdc[22948](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:45:25 dist-test-slave-pv0w krb5kdc[22948](info): set up 2 sockets
Sep 28 14:45:25 dist-test-slave-pv0w krb5kdc[22948](info): commencing operation
krb5kdc: starting...
W20260928 14:45:26.180608 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:45:26 dist-test-slave-pv0w krb5kdc[22948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:45:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:45:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.33.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
I20260928 14:45:26.306348 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:40131
--webserver_interface=127.9.33.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:37767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:40131
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:45:26.390687 22961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:45:26.391039 22961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:45:26.391085 22961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:45:26.393301 22961 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:45:26.393391 22961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:45:26.393419 22961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:45:26.393455 22961 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:45:26.393473 22961 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:45:26.395388 22961 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:37767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:40131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:40131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:45:26.395684 22961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:45:26.395972 22961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:45:26.401228 22967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:26.401309 22969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:26.401237 22966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:45:26.401372 22961 server_base.cc:1034] running on GCE node
I20260928 14:45:26.401894 22961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:45:26.402220 22961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:45:26.403406 22961 hybrid_clock.cc:648] HybridClock initialized: now 1790606726403390 us; error 45 us; skew 500 ppm
Sep 28 14:45:26 dist-test-slave-pv0w krb5kdc[22948](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606726, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:45:26.406114 22961 init.cc:377] Logged in from keytab as kudu/127.9.33.126@KRBTEST.COM (short username kudu)
I20260928 14:45:26.406694 22961 webserver.cc:460] Webserver started at http://127.9.33.126:38459/ using document root <none> and password file <none>
I20260928 14:45:26.406929 22961 fs_manager.cc:359] Metadata directory not provided
I20260928 14:45:26.406997 22961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:45:26.407136 22961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:45:26.408365 22961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "8df8ea37ab9b4d559f595d4098f39c96"
format_stamp: "Formatted at 2026-09-28 14:45:26 on dist-test-slave-pv0w"
server_key: "c6b68ae5ce090334f051a9e84b959047"
server_key_iv: "4b4c041bfefeef04949781d302cab5a0"
server_key_version: "encryptionkey@0"
I20260928 14:45:26.408742 22961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "8df8ea37ab9b4d559f595d4098f39c96"
format_stamp: "Formatted at 2026-09-28 14:45:26 on dist-test-slave-pv0w"
server_key: "c6b68ae5ce090334f051a9e84b959047"
server_key_iv: "4b4c041bfefeef04949781d302cab5a0"
server_key_version: "encryptionkey@0"
I20260928 14:45:26.410177 22961 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:45:26.410977 22976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:26.411204 22961 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:45:26.411377 22961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "8df8ea37ab9b4d559f595d4098f39c96"
format_stamp: "Formatted at 2026-09-28 14:45:26 on dist-test-slave-pv0w"
server_key: "c6b68ae5ce090334f051a9e84b959047"
server_key_iv: "4b4c041bfefeef04949781d302cab5a0"
server_key_version: "encryptionkey@0"
I20260928 14:45:26.411485 22961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:45:26.424341 22961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:45:26.426556 22961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:45:26.426738 22961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:45:26.431152 22961 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:40131
I20260928 14:45:26.431190 23028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:40131 every 8 connection(s)
I20260928 14:45:26.431622 22961 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:45:26.432540 23029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:45:26.434842 23029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8df8ea37ab9b4d559f595d4098f39c96: Bootstrap starting.
I20260928 14:45:26.435388 23029 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8df8ea37ab9b4d559f595d4098f39c96: Neither blocks nor log segments found. Creating new log.
I20260928 14:45:26.435626 23029 log.cc:824] T 00000000000000000000000000000000 P 8df8ea37ab9b4d559f595d4098f39c96: Log is configured to *not* fsync() on all Append() calls
I20260928 14:45:26.436480 23029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8df8ea37ab9b4d559f595d4098f39c96: No bootstrap required, opened a new log
I20260928 14:45:26.438136 23029 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8df8ea37ab9b4d559f595d4098f39c96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8df8ea37ab9b4d559f595d4098f39c96" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40131 } }
I20260928 14:45:26.438318 23029 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8df8ea37ab9b4d559f595d4098f39c96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:45:26.438345 23029 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8df8ea37ab9b4d559f595d4098f39c96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df8ea37ab9b4d559f595d4098f39c96, State: Initialized, Role: FOLLOWER
I20260928 14:45:26.438510 23029 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8df8ea37ab9b4d559f595d4098f39c96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8df8ea37ab9b4d559f595d4098f39c96" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40131 } }
I20260928 14:45:26.438598 23029 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8df8ea37ab9b4d559f595d4098f39c96 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:45:26.438623 23029 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8df8ea37ab9b4d559f595d4098f39c96 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:45:26.438663 23029 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8df8ea37ab9b4d559f595d4098f39c96 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:45:26.439502 23029 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8df8ea37ab9b4d559f595d4098f39c96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8df8ea37ab9b4d559f595d4098f39c96" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40131 } }
I20260928 14:45:26.439605 23029 leader_election.cc:302] T 00000000000000000000000000000000 P 8df8ea37ab9b4d559f595d4098f39c96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8df8ea37ab9b4d559f595d4098f39c96; no voters:
I20260928 14:45:26.439819 23029 leader_election.cc:288] T 00000000000000000000000000000000 P 8df8ea37ab9b4d559f595d4098f39c96 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:45:26.439893 23032 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8df8ea37ab9b4d559f595d4098f39c96 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:45:26.440078 23032 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8df8ea37ab9b4d559f595d4098f39c96 [term 1 LEADER]: Becoming Leader. State: Replica: 8df8ea37ab9b4d559f595d4098f39c96, State: Running, Role: LEADER
I20260928 14:45:26.440109 23029 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8df8ea37ab9b4d559f595d4098f39c96 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:45:26.440173 23032 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8df8ea37ab9b4d559f595d4098f39c96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8df8ea37ab9b4d559f595d4098f39c96" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40131 } }
I20260928 14:45:26.440791 23033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8df8ea37ab9b4d559f595d4098f39c96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8df8ea37ab9b4d559f595d4098f39c96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8df8ea37ab9b4d559f595d4098f39c96" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40131 } } }
I20260928 14:45:26.440897 23033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8df8ea37ab9b4d559f595d4098f39c96 [sys.catalog]: This master's current role is: LEADER
I20260928 14:45:26.441293 23041 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:45:26.441362 23034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8df8ea37ab9b4d559f595d4098f39c96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8df8ea37ab9b4d559f595d4098f39c96. Latest consensus state: current_term: 1 leader_uuid: "8df8ea37ab9b4d559f595d4098f39c96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8df8ea37ab9b4d559f595d4098f39c96" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40131 } } }
I20260928 14:45:26.441427 23034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8df8ea37ab9b4d559f595d4098f39c96 [sys.catalog]: This master's current role is: LEADER
I20260928 14:45:26.441715 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 22961
I20260928 14:45:26.441826 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:45:26.441849 23041 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:45:26.442039 9349 external_mini_cluster.cc:1428] Setting key ec9ca0cfe423291eda7b83c261bfba6d
I20260928 14:45:26.443755 23041 catalog_manager.cc:1347] Generated new cluster ID: 6c7a93a326484b589f3f071bf69f3c70
I20260928 14:45:26.443809 23041 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:45:26.448956 23041 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:45:26.449594 23041 catalog_manager.cc:1504] Loading token signing keys...
Sep 28 14:45:26 dist-test-slave-pv0w krb5kdc[22948](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:45:26.454571 23051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:26.442521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55924 (local address 127.9.33.126:40131)
0928 14:45:26.442705 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:26.442715 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:26.442808 (+ 93us) server_negotiation.cc:407] Connection header received
0928 14:45:26.442956 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:26.442964 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:26.443019 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:26.443185 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:26.444111 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:26.445561 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:26.446674 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:26.447355 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:26.447459 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:26.450549 (+ 3090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:26.450555 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:26.450556 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:26.450572 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:26.452682 (+ 2110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:26.453096 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:26.453099 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:26.453100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:26.453218 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:26.453424 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:26.453425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:26.453425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:26.453720 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:26.453888 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:26.454147 (+ 259us) server_negotiation.cc:299] Negotiation successful
0928 14:45:26.454316 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":70,"threads_started":1}
I20260928 14:45:26.456210 9349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:45:26.457129 23041 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8df8ea37ab9b4d559f595d4098f39c96: Generated new TSK 0
I20260928 14:45:26.457370 23041 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:45:26.510203 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 22961
2026-09-28T14:45:26Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1375 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:45:26 dist-test-slave-pv0w krb5kdc[23056](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:45:26 dist-test-slave-pv0w krb5kdc[23056](info): set up 2 sockets
Sep 28 14:45:26 dist-test-slave-pv0w krb5kdc[23056](info): commencing operation
krb5kdc: starting...
W20260928 14:45:27.544596 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:45:27 dist-test-slave-pv0w krb5kdc[23056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:45:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:45:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.33.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
I20260928 14:45:27.670733 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:46763
--webserver_interface=127.9.33.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:35485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:46763
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:45:27.753803 23069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:45:27.754108 23069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:45:27.754155 23069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:45:27.755425 23069 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:45:27.755502 23069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:45:27.755527 23069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:45:27.755558 23069 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:45:27.755585 23069 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:45:27.757169 23069 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:35485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:46763
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:46763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:45:27.757478 23069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:45:27.757721 23069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:45:27.762730 23075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:27.762756 23074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:27.762730 23077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:45:27.762952 23069 server_base.cc:1034] running on GCE node
I20260928 14:45:27.763244 23069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:45:27.763553 23069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:45:27.764716 23069 hybrid_clock.cc:648] HybridClock initialized: now 1790606727764696 us; error 47 us; skew 500 ppm
Sep 28 14:45:27 dist-test-slave-pv0w krb5kdc[23056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606727, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:45:27.767256 23069 init.cc:377] Logged in from keytab as kudu/127.9.33.126@KRBTEST.COM (short username kudu)
I20260928 14:45:27.767822 23069 webserver.cc:460] Webserver started at http://127.9.33.126:46303/ using document root <none> and password file <none>
I20260928 14:45:27.768069 23069 fs_manager.cc:359] Metadata directory not provided
I20260928 14:45:27.768137 23069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:45:27.768286 23069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:45:27.769423 23069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "ad4350e8668941ea9fa505f99e327043"
format_stamp: "Formatted at 2026-09-28 14:45:27 on dist-test-slave-pv0w"
server_key: "1e3c3ed5b194beb1c194636d9803ebf3"
server_key_iv: "3fc69158bdc24fcfd5336afb59720978"
server_key_version: "encryptionkey@0"
I20260928 14:45:27.769799 23069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "ad4350e8668941ea9fa505f99e327043"
format_stamp: "Formatted at 2026-09-28 14:45:27 on dist-test-slave-pv0w"
server_key: "1e3c3ed5b194beb1c194636d9803ebf3"
server_key_iv: "3fc69158bdc24fcfd5336afb59720978"
server_key_version: "encryptionkey@0"
I20260928 14:45:27.771436 23069 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:45:27.772312 23084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:27.772590 23069 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260928 14:45:27.772692 23069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "ad4350e8668941ea9fa505f99e327043"
format_stamp: "Formatted at 2026-09-28 14:45:27 on dist-test-slave-pv0w"
server_key: "1e3c3ed5b194beb1c194636d9803ebf3"
server_key_iv: "3fc69158bdc24fcfd5336afb59720978"
server_key_version: "encryptionkey@0"
I20260928 14:45:27.772775 23069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:45:27.794948 23069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:45:27.797183 23069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:45:27.797371 23069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:45:27.801494 23069 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:46763
I20260928 14:45:27.801532 23136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:46763 every 8 connection(s)
I20260928 14:45:27.801967 23069 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:45:27.803119 23137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:45:27.805790 23137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043: Bootstrap starting.
I20260928 14:45:27.806005 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 23069
I20260928 14:45:27.806130 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:45:27.806354 9349 external_mini_cluster.cc:1428] Setting key 341614ff9bbe949bebbe4947b229c1d9
I20260928 14:45:27.806562 23137 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043: Neither blocks nor log segments found. Creating new log.
I20260928 14:45:27.806927 23137 log.cc:824] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043: Log is configured to *not* fsync() on all Append() calls
I20260928 14:45:27.807778 23137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043: No bootstrap required, opened a new log
I20260928 14:45:27.810171 23137 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad4350e8668941ea9fa505f99e327043" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 46763 } }
I20260928 14:45:27.810382 23137 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:45:27.810463 23137 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad4350e8668941ea9fa505f99e327043, State: Initialized, Role: FOLLOWER
I20260928 14:45:27.810611 23137 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad4350e8668941ea9fa505f99e327043" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 46763 } }
I20260928 14:45:27.810688 23137 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:45:27.810729 23137 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:45:27.810786 23137 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:45:27.811481 23137 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad4350e8668941ea9fa505f99e327043" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 46763 } }
I20260928 14:45:27.811591 23137 leader_election.cc:302] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad4350e8668941ea9fa505f99e327043; no voters:
I20260928 14:45:27.811789 23137 leader_election.cc:288] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:45:27.811839 23142 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:45:27.811996 23142 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [term 1 LEADER]: Becoming Leader. State: Replica: ad4350e8668941ea9fa505f99e327043, State: Running, Role: LEADER
I20260928 14:45:27.812067 23137 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:45:27.812101 23142 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad4350e8668941ea9fa505f99e327043" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 46763 } }
I20260928 14:45:27.812459 23144 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad4350e8668941ea9fa505f99e327043" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad4350e8668941ea9fa505f99e327043" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 46763 } } }
I20260928 14:45:27.812449 23142 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad4350e8668941ea9fa505f99e327043. Latest consensus state: current_term: 1 leader_uuid: "ad4350e8668941ea9fa505f99e327043" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad4350e8668941ea9fa505f99e327043" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 46763 } } }
I20260928 14:45:27.812572 23144 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [sys.catalog]: This master's current role is: LEADER
I20260928 14:45:27.812592 23142 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [sys.catalog]: This master's current role is: LEADER
I20260928 14:45:27.812906 23150 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:45:27.813477 23150 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 28 14:45:27 dist-test-slave-pv0w krb5kdc[23056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:45:27.815574 23150 catalog_manager.cc:1347] Generated new cluster ID: d38f0742c7664947ac2e98492bc58dbd
I20260928 14:45:27.815626 23150 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:45:27.819432 23140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:27.806947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41068 (local address 127.9.33.126:46763)
0928 14:45:27.807155 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:27.807165 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:27.807206 (+ 41us) server_negotiation.cc:407] Connection header received
0928 14:45:27.807845 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:27.807855 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:27.807933 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:27.808096 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:27.808767 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:27.810239 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:27.811259 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:27.811966 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:27.812075 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:27.815374 (+ 3299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:27.815379 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:27.815380 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:27.815393 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:27.817650 (+ 2257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:27.818092 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:27.818095 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:27.818096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:27.818219 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:27.818426 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:27.818427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:27.818427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:27.818769 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:27.818928 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:27.819043 (+ 115us) server_negotiation.cc:299] Negotiation successful
0928 14:45:27.819203 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":93,"threads_started":1}
I20260928 14:45:27.826078 23150 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:45:27.826687 23150 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:45:27.833093 23150 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043: Generated new TSK 0
I20260928 14:45:27.833319 23150 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.33.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab.
Entry for principal kudu/127.9.33.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab.
I20260928 14:45:27.864084 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.65:0
--local_ip_for_outbound_sockets=127.9.33.65
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:46763
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:35485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:45:27.952266 23163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:45:27.952491 23163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:45:27.952517 23163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:45:27.953649 23163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:45:27.953691 23163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:45:27.953718 23163 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.65
I20260928 14:45:27.955279 23163 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:35485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:46763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.65
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:45:27.955510 23163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:45:27.955705 23163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:45:27.961257 23169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:27.961297 23171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:45:27.961437 23163 server_base.cc:1034] running on GCE node
W20260928 14:45:27.961279 23168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:45:27.961728 23163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:45:27.962007 23163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:45:27.963181 23163 hybrid_clock.cc:648] HybridClock initialized: now 1790606727963161 us; error 41 us; skew 500 ppm
Sep 28 14:45:27 dist-test-slave-pv0w krb5kdc[23056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606727, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:45:27.965873 23163 init.cc:377] Logged in from keytab as kudu/127.9.33.65@KRBTEST.COM (short username kudu)
I20260928 14:45:27.966369 23163 webserver.cc:460] Webserver started at http://127.9.33.65:46089/ using document root <none> and password file <none>
I20260928 14:45:27.966578 23163 fs_manager.cc:359] Metadata directory not provided
I20260928 14:45:27.966620 23163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:45:27.966715 23163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:45:27.967844 23163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/data/instance:
uuid: "fa74e6c10d9241628165ee829514df21"
format_stamp: "Formatted at 2026-09-28 14:45:27 on dist-test-slave-pv0w"
server_key: "6d2358cd2f05abf78e5a16810320fbb7"
server_key_iv: "f5cb5c9ee5ac786dda0ec001022df31f"
server_key_version: "encryptionkey@0"
I20260928 14:45:27.968218 23163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance:
uuid: "fa74e6c10d9241628165ee829514df21"
format_stamp: "Formatted at 2026-09-28 14:45:27 on dist-test-slave-pv0w"
server_key: "6d2358cd2f05abf78e5a16810320fbb7"
server_key_iv: "f5cb5c9ee5ac786dda0ec001022df31f"
server_key_version: "encryptionkey@0"
I20260928 14:45:27.969595 23163 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260928 14:45:27.970423 23178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:27.970638 23163 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:45:27.970742 23163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/wal
uuid: "fa74e6c10d9241628165ee829514df21"
format_stamp: "Formatted at 2026-09-28 14:45:27 on dist-test-slave-pv0w"
server_key: "6d2358cd2f05abf78e5a16810320fbb7"
server_key_iv: "f5cb5c9ee5ac786dda0ec001022df31f"
server_key_version: "encryptionkey@0"
I20260928 14:45:27.970805 23163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:45:27.982265 23163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:45:27.984656 23163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:45:27.984787 23163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:45:27.984982 23163 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:45:27.985312 23163 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:45:27.985347 23163 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:27.985392 23163 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:45:27.985419 23163 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:27.991138 23163 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.65:33867
I20260928 14:45:27.991221 23291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.65:33867 every 8 connection(s)
I20260928 14:45:27.991564 23163 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
Sep 28 14:45:27 dist-test-slave-pv0w krb5kdc[23056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606727, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:45:27.999773 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 23163
I20260928 14:45:27.999895 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance
I20260928 14:45:28.000120 9349 external_mini_cluster.cc:1428] Setting key 470972e7052f81dda4703cab290ad19d
I20260928 14:45:28.002139 23140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:27.992570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:57941 (local address 127.9.33.126:46763)
0928 14:45:27.992659 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:27.992661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:27.992878 (+ 217us) server_negotiation.cc:407] Connection header received
0928 14:45:27.993374 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:27.993375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:27.993386 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:27.993454 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:27.994478 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:27.994990 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:27.996200 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:27.996415 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:27.996477 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:27.998793 (+ 2316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:27.998795 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:27.998795 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:27.998804 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:28.000582 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:28.001053 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:28.001054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:28.001055 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:28.001123 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:28.001446 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:28.001447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:28.001447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:28.001575 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:28.001652 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:28.001943 (+ 291us) server_negotiation.cc:299] Negotiation successful
0928 14:45:28.002039 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260928 14:45:28.002450 23294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:27.992497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.65:57941)
0928 14:45:27.992730 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:27.992785 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:27.993261 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:27.993534 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:27.993538 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:27.994164 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:45:27.994375 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:27.994380 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:27.995077 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:27.995078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:27.996082 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:27.996085 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:27.996497 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:27.996498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:27.996618 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:27.997140 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:45:27.997150 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:45:27.998686 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:45:28.000678 (+ 1992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:28.000689 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:28.000692 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:28.000958 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:28.001193 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:28.001195 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:28.001195 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:28.001372 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:28.001644 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:45:28.001653 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:45:28.001809 (+ 156us) client_negotiation.cc:769] Sending connection context
0928 14:45:28.001924 (+ 115us) client_negotiation.cc:240] Negotiation successful
0928 14:45:28.002105 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":102,"threads_started":1}
I20260928 14:45:28.002961 23292 heartbeater.cc:347] Connected to a master server at 127.9.33.126:46763
I20260928 14:45:28.003074 23292 heartbeater.cc:464] Registering TS with master...
I20260928 14:45:28.003263 23292 heartbeater.cc:511] Master 127.9.33.126:46763 requested a full tablet report, sending...
I20260928 14:45:28.003882 23101 ts_manager.cc:194] Registered new tserver with Master: fa74e6c10d9241628165ee829514df21 (127.9.33.65:33867)
I20260928 14:45:28.004520 23101 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.65@KRBTEST.COM'} at 127.9.33.65:57941
WARNING: no policy specified for kudu/127.9.33.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab.
Entry for principal kudu/127.9.33.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab.
I20260928 14:45:28.031227 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.66:0
--local_ip_for_outbound_sockets=127.9.33.66
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:46763
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:35485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:45:28.115212 23297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:45:28.115428 23297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:45:28.115454 23297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:45:28.116808 23297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:45:28.116854 23297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:45:28.116883 23297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.66
I20260928 14:45:28.118388 23297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:35485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:46763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.66
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:45:28.118650 23297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:45:28.118918 23297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:45:28.124244 23302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:28.124234 23305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:45:28.124284 23297 server_base.cc:1034] running on GCE node
W20260928 14:45:28.124235 23303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:45:28.124693 23297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:45:28.124914 23297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:45:28.126076 23297 hybrid_clock.cc:648] HybridClock initialized: now 1790606728126054 us; error 36 us; skew 500 ppm
Sep 28 14:45:28 dist-test-slave-pv0w krb5kdc[23056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606728, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:45:28.129026 23297 init.cc:377] Logged in from keytab as kudu/127.9.33.66@KRBTEST.COM (short username kudu)
I20260928 14:45:28.129558 23297 webserver.cc:460] Webserver started at http://127.9.33.66:44581/ using document root <none> and password file <none>
I20260928 14:45:28.129817 23297 fs_manager.cc:359] Metadata directory not provided
I20260928 14:45:28.129864 23297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:45:28.130048 23297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:45:28.131204 23297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/data/instance:
uuid: "90ef635255234a8d8998cfa7951f1e32"
format_stamp: "Formatted at 2026-09-28 14:45:28 on dist-test-slave-pv0w"
server_key: "8ccbe6dc6913e0350f1ccc4933697848"
server_key_iv: "5ed48d52c22b6c84de3afc350ee33f4c"
server_key_version: "encryptionkey@0"
I20260928 14:45:28.131587 23297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance:
uuid: "90ef635255234a8d8998cfa7951f1e32"
format_stamp: "Formatted at 2026-09-28 14:45:28 on dist-test-slave-pv0w"
server_key: "8ccbe6dc6913e0350f1ccc4933697848"
server_key_iv: "5ed48d52c22b6c84de3afc350ee33f4c"
server_key_version: "encryptionkey@0"
I20260928 14:45:28.133109 23297 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:45:28.133946 23312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:28.134182 23297 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:45:28.134298 23297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/wal
uuid: "90ef635255234a8d8998cfa7951f1e32"
format_stamp: "Formatted at 2026-09-28 14:45:28 on dist-test-slave-pv0w"
server_key: "8ccbe6dc6913e0350f1ccc4933697848"
server_key_iv: "5ed48d52c22b6c84de3afc350ee33f4c"
server_key_version: "encryptionkey@0"
I20260928 14:45:28.134394 23297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:45:28.157109 23297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:45:28.159260 23297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:45:28.159408 23297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:45:28.159665 23297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:45:28.160037 23297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:45:28.160071 23297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:28.160095 23297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:45:28.160108 23297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:28.166046 23297 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.66:38363
I20260928 14:45:28.166146 23425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.66:38363 every 8 connection(s)
I20260928 14:45:28.166448 23297 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
I20260928 14:45:28.166752 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 23297
I20260928 14:45:28.166872 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance
I20260928 14:45:28.167104 9349 external_mini_cluster.cc:1428] Setting key a6e1ccf64339ca1f2536e66319435262
Sep 28 14:45:28 dist-test-slave-pv0w krb5kdc[23056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606728, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.33.67@KRBTEST.COM; defaulting to no policy
I20260928 14:45:28.177253 23140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:28.167571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:58503 (local address 127.9.33.126:46763)
0928 14:45:28.167621 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:28.167624 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:28.167634 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:45:28.168208 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:28.168209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:28.168223 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:28.168300 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:28.169281 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.169794 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.170810 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.171030 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.171114 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:28.173771 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:28.173773 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:28.173774 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:28.173782 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:28.175514 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:28.176066 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:28.176068 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:28.176068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:28.176132 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:28.176426 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:28.176427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:28.176428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:28.176601 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:28.176676 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:28.176987 (+ 311us) server_negotiation.cc:299] Negotiation successful
0928 14:45:28.177096 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 14:45:28.177403 23428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:28.167327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.66:58503)
0928 14:45:28.167493 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:28.167536 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:28.168099 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:28.168379 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:28.168383 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:28.168967 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:45:28.169171 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:28.169177 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.169892 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.169893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:28.170711 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:28.170713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.171126 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.171127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:28.171258 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:28.171851 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:45:28.171861 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:45:28.173590 (+ 1729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:45:28.175592 (+ 2002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:28.175602 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:28.175605 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:28.175953 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:28.176227 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:28.176228 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:28.176229 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:28.176353 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:28.176666 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:45:28.176674 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:45:28.176845 (+ 171us) client_negotiation.cc:769] Sending connection context
0928 14:45:28.176950 (+ 105us) client_negotiation.cc:240] Negotiation successful
0928 14:45:28.177149 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":88,"threads_started":1}
I20260928 14:45:28.177994 23426 heartbeater.cc:347] Connected to a master server at 127.9.33.126:46763
I20260928 14:45:28.178128 23426 heartbeater.cc:464] Registering TS with master...
I20260928 14:45:28.178351 23426 heartbeater.cc:511] Master 127.9.33.126:46763 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.67@KRBTEST.COM" created.
I20260928 14:45:28.178879 23101 ts_manager.cc:194] Registered new tserver with Master: 90ef635255234a8d8998cfa7951f1e32 (127.9.33.66:38363)
I20260928 14:45:28.179309 23101 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.66@KRBTEST.COM'} at 127.9.33.66:58503
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab.
Entry for principal kudu/127.9.33.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab.
I20260928 14:45:28.196226 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.67:0
--local_ip_for_outbound_sockets=127.9.33.67
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:46763
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:35485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:45:28.284514 23431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:45:28.284798 23431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:45:28.284827 23431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:45:28.286160 23431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:45:28.286212 23431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:45:28.286240 23431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.67
I20260928 14:45:28.287880 23431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:35485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:46763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.67
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:45:28.288179 23431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:45:28.288460 23431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:45:28.294260 23436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:28.294279 23439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:28.294260 23437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:45:28.294540 23431 server_base.cc:1034] running on GCE node
I20260928 14:45:28.294736 23431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:45:28.295012 23431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:45:28.296169 23431 hybrid_clock.cc:648] HybridClock initialized: now 1790606728296150 us; error 41 us; skew 500 ppm
Sep 28 14:45:28 dist-test-slave-pv0w krb5kdc[23056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606728, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:45:28.298964 23431 init.cc:377] Logged in from keytab as kudu/127.9.33.67@KRBTEST.COM (short username kudu)
I20260928 14:45:28.299513 23431 webserver.cc:460] Webserver started at http://127.9.33.67:46637/ using document root <none> and password file <none>
I20260928 14:45:28.299844 23431 fs_manager.cc:359] Metadata directory not provided
I20260928 14:45:28.299929 23431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:45:28.300100 23431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:45:28.301242 23431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/data/instance:
uuid: "b43bdfad8d3b4aa1bf499e9148871c0e"
format_stamp: "Formatted at 2026-09-28 14:45:28 on dist-test-slave-pv0w"
server_key: "01e19ff0a4f3acc9e3040683e1657e1d"
server_key_iv: "e35e4b450a85b2eae000a3c8b2da56df"
server_key_version: "encryptionkey@0"
I20260928 14:45:28.301769 23431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance:
uuid: "b43bdfad8d3b4aa1bf499e9148871c0e"
format_stamp: "Formatted at 2026-09-28 14:45:28 on dist-test-slave-pv0w"
server_key: "01e19ff0a4f3acc9e3040683e1657e1d"
server_key_iv: "e35e4b450a85b2eae000a3c8b2da56df"
server_key_version: "encryptionkey@0"
I20260928 14:45:28.303534 23431 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:45:28.304701 23446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:28.304934 23431 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:45:28.305054 23431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/wal
uuid: "b43bdfad8d3b4aa1bf499e9148871c0e"
format_stamp: "Formatted at 2026-09-28 14:45:28 on dist-test-slave-pv0w"
server_key: "01e19ff0a4f3acc9e3040683e1657e1d"
server_key_iv: "e35e4b450a85b2eae000a3c8b2da56df"
server_key_version: "encryptionkey@0"
I20260928 14:45:28.305158 23431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:45:28.317633 23431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:45:28.320263 23431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:45:28.320446 23431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:45:28.320703 23431 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:45:28.321038 23431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:45:28.321094 23431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:28.321141 23431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:45:28.321171 23431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:28.327864 23431 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.67:45783
I20260928 14:45:28.327903 23559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.67:45783 every 8 connection(s)
I20260928 14:45:28.328286 23431 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
I20260928 14:45:28.331995 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 23431
I20260928 14:45:28.332100 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance
I20260928 14:45:28.332288 9349 external_mini_cluster.cc:1428] Setting key 2bcbb5da8ed986e3c92e2ca9cb4f5437
Sep 28 14:45:28 dist-test-slave-pv0w krb5kdc[23056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606728, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:45:28.340503 23140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:28.329174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:40731 (local address 127.9.33.126:46763)
0928 14:45:28.329332 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:28.329381 (+ 49us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:28.329526 (+ 145us) server_negotiation.cc:407] Connection header received
0928 14:45:28.330717 (+ 1191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:28.330719 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:28.330735 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:28.330787 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:28.331765 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.332629 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.334053 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.334286 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.334371 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:28.336963 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:28.336966 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:28.336966 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:28.336974 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:28.338826 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:28.339326 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:28.339328 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:28.339329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:28.339397 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:28.339706 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:28.339707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:28.339707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:28.339892 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:28.339973 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:28.340290 (+ 317us) server_negotiation.cc:299] Negotiation successful
0928 14:45:28.340393 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260928 14:45:28.340734 23562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:28.329119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.67:40731)
0928 14:45:28.329363 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:28.329428 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:28.330485 (+ 1057us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:28.330872 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:28.330877 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:28.331205 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:45:28.331440 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:28.331445 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.332752 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.332753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:28.333953 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:28.333957 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.334384 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.334386 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:28.334541 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:28.335139 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:45:28.335169 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:45:28.336826 (+ 1657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:45:28.338936 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:28.338948 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:28.338951 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:28.339236 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:28.339474 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:28.339475 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:28.339476 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:28.339624 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:28.339975 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:45:28.339985 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:45:28.340150 (+ 165us) client_negotiation.cc:769] Sending connection context
0928 14:45:28.340269 (+ 119us) client_negotiation.cc:240] Negotiation successful
0928 14:45:28.340477 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":107,"threads_started":1}
I20260928 14:45:28.341416 23560 heartbeater.cc:347] Connected to a master server at 127.9.33.126:46763
I20260928 14:45:28.341517 23560 heartbeater.cc:464] Registering TS with master...
I20260928 14:45:28.341769 23560 heartbeater.cc:511] Master 127.9.33.126:46763 requested a full tablet report, sending...
I20260928 14:45:28.342324 23101 ts_manager.cc:194] Registered new tserver with Master: b43bdfad8d3b4aa1bf499e9148871c0e (127.9.33.67:45783)
I20260928 14:45:28.342760 23101 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.67@KRBTEST.COM'} at 127.9.33.67:40731
I20260928 14:45:28.344424 9349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:45:28.352339 23140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:28.345888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41078 (local address 127.9.33.126:46763)
0928 14:45:28.345950 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:28.345952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:28.346001 (+ 49us) server_negotiation.cc:407] Connection header received
0928 14:45:28.346055 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:28.346056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:28.346064 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:28.346119 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:28.346753 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.347259 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.348134 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.348331 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.348390 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:28.349629 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:28.349630 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:28.349631 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:28.349637 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:28.351131 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:28.351491 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:28.351492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:28.351493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:28.351548 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:28.351834 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:28.351834 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:28.351835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:28.352004 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:28.352090 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:28.352194 (+ 104us) server_negotiation.cc:299] Negotiation successful
0928 14:45:28.352259 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260928 14:45:28.354270 23101 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41078:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:45:28.356112 23101 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:45:28.363497 23572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:28.358711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:33867 (local address 127.0.0.1:49324)
0928 14:45:28.359134 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:28.359147 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:28.359214 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:28.359543 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:28.359547 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:28.359550 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:45:28.359631 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:28.359633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.361058 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.361059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:28.362174 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:28.362177 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.363115 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.363117 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:28.363221 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:28.363243 (+ 22us) client_negotiation.cc:769] Sending connection context
0928 14:45:28.363320 (+ 77us) client_negotiation.cc:240] Negotiation successful
0928 14:45:28.363392 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"mutex_wait_us":73,"thread_start_us":101,"threads_started":1}
I20260928 14:45:28.363596 23576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:28.358989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32994 (local address 127.9.33.66:38363)
0928 14:45:28.359819 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:28.359823 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:28.359833 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:45:28.359872 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:28.359874 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:28.359888 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:28.359988 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:45:28.360090 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.360817 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.362578 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.363193 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.363268 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:28.363336 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:28.363460 (+ 124us) server_negotiation.cc:299] Negotiation successful
0928 14:45:28.363501 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":789,"thread_start_us":79,"threads_started":1}
I20260928 14:45:28.363922 23574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:28.358901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35862 (local address 127.9.33.67:45783)
0928 14:45:28.359076 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:28.359082 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:28.359376 (+ 294us) server_negotiation.cc:407] Connection header received
0928 14:45:28.359575 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:28.359578 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:28.359596 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:28.359690 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:45:28.359840 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.360947 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.362559 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.363316 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.363384 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:28.363440 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:28.363784 (+ 344us) server_negotiation.cc:299] Negotiation successful
0928 14:45:28.363832 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":80,"threads_started":1}
I20260928 14:45:28.364212 23575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:28.358930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49324 (local address 127.9.33.65:33867)
0928 14:45:28.359284 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:28.359288 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:28.359301 (+ 13us) server_negotiation.cc:407] Connection header received
0928 14:45:28.359349 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:28.359352 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:28.359370 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:28.359446 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:45:28.359756 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.360948 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.362272 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.362994 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.363075 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:28.363156 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:28.364049 (+ 893us) server_negotiation.cc:299] Negotiation successful
0928 14:45:28.364112 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":92,"threads_started":1}
I20260928 14:45:28.363672 23571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:28.358716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:45783 (local address 127.0.0.1:35862)
0928 14:45:28.359317 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:28.359325 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:28.359373 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:28.359693 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:28.359694 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:28.359696 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:45:28.359778 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:28.359780 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.361150 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.361150 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:28.362456 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:28.362459 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.363472 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.363472 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:28.363528 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:28.363531 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:45:28.363579 (+ 48us) client_negotiation.cc:240] Negotiation successful
0928 14:45:28.363614 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":114,"threads_started":1}
I20260928 14:45:28.363832 23573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:28.358694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:38363 (local address 127.0.0.1:32994)
0928 14:45:28.359099 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:28.359143 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:28.359214 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:28.359996 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:28.359998 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:28.360000 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:45:28.360039 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:28.360040 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.360936 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.360937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:28.362457 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:28.362459 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.363295 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.363297 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:28.363370 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:28.363375 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:45:28.363429 (+ 54us) client_negotiation.cc:240] Negotiation successful
0928 14:45:28.363482 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"mutex_wait_us":96,"thread_start_us":58,"threads_started":1}
I20260928 14:45:28.365000 23226 tablet_service.cc:1467] Processing CreateTablet for tablet c8ca0a9372314a7a90e2c0fe30f86155 (DEFAULT_TABLE table=test-table [id=141e4fd4dc9a44bf8f43852e002d1402]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:45:28.365101 23494 tablet_service.cc:1467] Processing CreateTablet for tablet c8ca0a9372314a7a90e2c0fe30f86155 (DEFAULT_TABLE table=test-table [id=141e4fd4dc9a44bf8f43852e002d1402]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:45:28.365136 23360 tablet_service.cc:1467] Processing CreateTablet for tablet c8ca0a9372314a7a90e2c0fe30f86155 (DEFAULT_TABLE table=test-table [id=141e4fd4dc9a44bf8f43852e002d1402]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:45:28.365379 23226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8ca0a9372314a7a90e2c0fe30f86155. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:45:28.365450 23360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8ca0a9372314a7a90e2c0fe30f86155. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:45:28.365450 23494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8ca0a9372314a7a90e2c0fe30f86155. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:45:28.368487 23577 tablet_bootstrap.cc:492] T c8ca0a9372314a7a90e2c0fe30f86155 P fa74e6c10d9241628165ee829514df21: Bootstrap starting.
I20260928 14:45:28.368899 23578 tablet_bootstrap.cc:492] T c8ca0a9372314a7a90e2c0fe30f86155 P 90ef635255234a8d8998cfa7951f1e32: Bootstrap starting.
I20260928 14:45:28.369638 23578 tablet_bootstrap.cc:654] T c8ca0a9372314a7a90e2c0fe30f86155 P 90ef635255234a8d8998cfa7951f1e32: Neither blocks nor log segments found. Creating new log.
I20260928 14:45:28.369638 23577 tablet_bootstrap.cc:654] T c8ca0a9372314a7a90e2c0fe30f86155 P fa74e6c10d9241628165ee829514df21: Neither blocks nor log segments found. Creating new log.
I20260928 14:45:28.370285 23578 log.cc:824] T c8ca0a9372314a7a90e2c0fe30f86155 P 90ef635255234a8d8998cfa7951f1e32: Log is configured to *not* fsync() on all Append() calls
I20260928 14:45:28.370289 23577 log.cc:824] T c8ca0a9372314a7a90e2c0fe30f86155 P fa74e6c10d9241628165ee829514df21: Log is configured to *not* fsync() on all Append() calls
I20260928 14:45:28.370668 23579 tablet_bootstrap.cc:492] T c8ca0a9372314a7a90e2c0fe30f86155 P b43bdfad8d3b4aa1bf499e9148871c0e: Bootstrap starting.
I20260928 14:45:28.371097 23578 tablet_bootstrap.cc:492] T c8ca0a9372314a7a90e2c0fe30f86155 P 90ef635255234a8d8998cfa7951f1e32: No bootstrap required, opened a new log
I20260928 14:45:28.371196 23578 ts_tablet_manager.cc:1397] T c8ca0a9372314a7a90e2c0fe30f86155 P 90ef635255234a8d8998cfa7951f1e32: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:45:28.371321 23579 tablet_bootstrap.cc:654] T c8ca0a9372314a7a90e2c0fe30f86155 P b43bdfad8d3b4aa1bf499e9148871c0e: Neither blocks nor log segments found. Creating new log.
I20260928 14:45:28.371886 23579 log.cc:824] T c8ca0a9372314a7a90e2c0fe30f86155 P b43bdfad8d3b4aa1bf499e9148871c0e: Log is configured to *not* fsync() on all Append() calls
I20260928 14:45:28.372392 23577 tablet_bootstrap.cc:492] T c8ca0a9372314a7a90e2c0fe30f86155 P fa74e6c10d9241628165ee829514df21: No bootstrap required, opened a new log
I20260928 14:45:28.372450 23577 ts_tablet_manager.cc:1397] T c8ca0a9372314a7a90e2c0fe30f86155 P fa74e6c10d9241628165ee829514df21: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260928 14:45:28.373123 23578 raft_consensus.cc:348] T c8ca0a9372314a7a90e2c0fe30f86155 P 90ef635255234a8d8998cfa7951f1e32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ef635255234a8d8998cfa7951f1e32" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38363 } } peers { permanent_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45783 } } peers { permanent_uuid: "fa74e6c10d9241628165ee829514df21" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33867 } }
I20260928 14:45:28.373322 23578 raft_consensus.cc:374] T c8ca0a9372314a7a90e2c0fe30f86155 P 90ef635255234a8d8998cfa7951f1e32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:45:28.373385 23578 raft_consensus.cc:729] T c8ca0a9372314a7a90e2c0fe30f86155 P 90ef635255234a8d8998cfa7951f1e32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90ef635255234a8d8998cfa7951f1e32, State: Initialized, Role: FOLLOWER
I20260928 14:45:28.373550 23578 consensus_queue.cc:260] T c8ca0a9372314a7a90e2c0fe30f86155 P 90ef635255234a8d8998cfa7951f1e32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ef635255234a8d8998cfa7951f1e32" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38363 } } peers { permanent_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45783 } } peers { permanent_uuid: "fa74e6c10d9241628165ee829514df21" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33867 } }
I20260928 14:45:28.373901 23578 ts_tablet_manager.cc:1428] T c8ca0a9372314a7a90e2c0fe30f86155 P 90ef635255234a8d8998cfa7951f1e32: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:45:28.373899 23426 heartbeater.cc:503] Master 127.9.33.126:46763 was elected leader, sending a full tablet report...
I20260928 14:45:28.374271 23577 raft_consensus.cc:348] T c8ca0a9372314a7a90e2c0fe30f86155 P fa74e6c10d9241628165ee829514df21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ef635255234a8d8998cfa7951f1e32" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38363 } } peers { permanent_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45783 } } peers { permanent_uuid: "fa74e6c10d9241628165ee829514df21" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33867 } }
I20260928 14:45:28.374455 23577 raft_consensus.cc:374] T c8ca0a9372314a7a90e2c0fe30f86155 P fa74e6c10d9241628165ee829514df21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:45:28.374503 23577 raft_consensus.cc:729] T c8ca0a9372314a7a90e2c0fe30f86155 P fa74e6c10d9241628165ee829514df21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa74e6c10d9241628165ee829514df21, State: Initialized, Role: FOLLOWER
I20260928 14:45:28.374607 23577 consensus_queue.cc:260] T c8ca0a9372314a7a90e2c0fe30f86155 P fa74e6c10d9241628165ee829514df21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ef635255234a8d8998cfa7951f1e32" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38363 } } peers { permanent_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45783 } } peers { permanent_uuid: "fa74e6c10d9241628165ee829514df21" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33867 } }
I20260928 14:45:28.374912 23577 ts_tablet_manager.cc:1428] T c8ca0a9372314a7a90e2c0fe30f86155 P fa74e6c10d9241628165ee829514df21: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260928 14:45:28.374923 23292 heartbeater.cc:503] Master 127.9.33.126:46763 was elected leader, sending a full tablet report...
I20260928 14:45:28.375738 23579 tablet_bootstrap.cc:492] T c8ca0a9372314a7a90e2c0fe30f86155 P b43bdfad8d3b4aa1bf499e9148871c0e: No bootstrap required, opened a new log
I20260928 14:45:28.375823 23579 ts_tablet_manager.cc:1397] T c8ca0a9372314a7a90e2c0fe30f86155 P b43bdfad8d3b4aa1bf499e9148871c0e: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260928 14:45:28.377388 23579 raft_consensus.cc:348] T c8ca0a9372314a7a90e2c0fe30f86155 P b43bdfad8d3b4aa1bf499e9148871c0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ef635255234a8d8998cfa7951f1e32" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38363 } } peers { permanent_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45783 } } peers { permanent_uuid: "fa74e6c10d9241628165ee829514df21" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33867 } }
I20260928 14:45:28.377545 23579 raft_consensus.cc:374] T c8ca0a9372314a7a90e2c0fe30f86155 P b43bdfad8d3b4aa1bf499e9148871c0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:45:28.377569 23579 raft_consensus.cc:729] T c8ca0a9372314a7a90e2c0fe30f86155 P b43bdfad8d3b4aa1bf499e9148871c0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b43bdfad8d3b4aa1bf499e9148871c0e, State: Initialized, Role: FOLLOWER
I20260928 14:45:28.377660 23579 consensus_queue.cc:260] T c8ca0a9372314a7a90e2c0fe30f86155 P b43bdfad8d3b4aa1bf499e9148871c0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ef635255234a8d8998cfa7951f1e32" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38363 } } peers { permanent_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45783 } } peers { permanent_uuid: "fa74e6c10d9241628165ee829514df21" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33867 } }
I20260928 14:45:28.377883 23579 ts_tablet_manager.cc:1428] T c8ca0a9372314a7a90e2c0fe30f86155 P b43bdfad8d3b4aa1bf499e9148871c0e: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260928 14:45:28.377940 23560 heartbeater.cc:503] Master 127.9.33.126:46763 was elected leader, sending a full tablet report...
I20260928 14:45:28.381958 23585 raft_consensus.cc:482] T c8ca0a9372314a7a90e2c0fe30f86155 P b43bdfad8d3b4aa1bf499e9148871c0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:45:28.382056 23585 raft_consensus.cc:504] T c8ca0a9372314a7a90e2c0fe30f86155 P b43bdfad8d3b4aa1bf499e9148871c0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ef635255234a8d8998cfa7951f1e32" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38363 } } peers { permanent_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45783 } } peers { permanent_uuid: "fa74e6c10d9241628165ee829514df21" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33867 } }
I20260928 14:45:28.382439 23585 leader_election.cc:288] T c8ca0a9372314a7a90e2c0fe30f86155 P b43bdfad8d3b4aa1bf499e9148871c0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90ef635255234a8d8998cfa7951f1e32 (127.9.33.66:38363), fa74e6c10d9241628165ee829514df21 (127.9.33.65:33867)
I20260928 14:45:28.386993 23576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:28.382890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:58879 (local address 127.9.33.66:38363)
0928 14:45:28.383015 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:28.383017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:28.383027 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:45:28.383056 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:28.383056 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:28.383068 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:28.383131 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:45:28.383452 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.384251 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.385903 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.386503 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.386596 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:28.386608 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:28.386871 (+ 263us) server_negotiation.cc:299] Negotiation successful
0928 14:45:28.386921 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260928 14:45:28.386993 23575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:28.382956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:53879 (local address 127.9.33.65:33867)
0928 14:45:28.383015 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:28.383017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:28.383027 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:45:28.383056 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:28.383057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:28.383068 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:28.383131 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:45:28.383452 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.384285 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.385904 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.386534 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.386599 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:28.386608 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:28.386871 (+ 263us) server_negotiation.cc:299] Negotiation successful
0928 14:45:28.386920 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260928 14:45:28.387020 23562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:28.382726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:33867 (local address 127.9.33.67:53879)
0928 14:45:28.382813 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:28.382827 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:28.382881 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:28.383220 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:28.383221 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:28.383224 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:45:28.383322 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:28.383324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.384371 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.384372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:28.385789 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:28.385792 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.386651 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.386652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:28.386750 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:28.386754 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:45:28.386832 (+ 78us) client_negotiation.cc:240] Negotiation successful
0928 14:45:28.386906 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":187}
I20260928 14:45:28.387030 23586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:28.382737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:38363 (local address 127.9.33.67:58879)
0928 14:45:28.382959 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:28.382975 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:28.383009 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:28.383220 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:28.383221 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:28.383224 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:45:28.383315 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:28.383317 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.384371 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.384372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:28.385788 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:28.385792 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.386651 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.386652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:28.386761 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:28.386764 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:45:28.386828 (+ 64us) client_negotiation.cc:240] Negotiation successful
0928 14:45:28.386907 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":59,"thread_start_us":145,"threads_started":1}
I20260928 14:45:28.387434 23246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8ca0a9372314a7a90e2c0fe30f86155" candidate_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa74e6c10d9241628165ee829514df21" is_pre_election: true
I20260928 14:45:28.387629 23246 raft_consensus.cc:2454] T c8ca0a9372314a7a90e2c0fe30f86155 P fa74e6c10d9241628165ee829514df21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b43bdfad8d3b4aa1bf499e9148871c0e in term 0.
I20260928 14:45:28.387874 23448 leader_election.cc:302] T c8ca0a9372314a7a90e2c0fe30f86155 P b43bdfad8d3b4aa1bf499e9148871c0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b43bdfad8d3b4aa1bf499e9148871c0e, fa74e6c10d9241628165ee829514df21; no voters:
I20260928 14:45:28.388016 23585 raft_consensus.cc:2781] T c8ca0a9372314a7a90e2c0fe30f86155 P b43bdfad8d3b4aa1bf499e9148871c0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:45:28.388094 23585 raft_consensus.cc:482] T c8ca0a9372314a7a90e2c0fe30f86155 P b43bdfad8d3b4aa1bf499e9148871c0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:45:28.388120 23585 raft_consensus.cc:3037] T c8ca0a9372314a7a90e2c0fe30f86155 P b43bdfad8d3b4aa1bf499e9148871c0e [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:45:28.388069 23380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8ca0a9372314a7a90e2c0fe30f86155" candidate_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90ef635255234a8d8998cfa7951f1e32" is_pre_election: true
I20260928 14:45:28.388199 23380 raft_consensus.cc:2454] T c8ca0a9372314a7a90e2c0fe30f86155 P 90ef635255234a8d8998cfa7951f1e32 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b43bdfad8d3b4aa1bf499e9148871c0e in term 0.
I20260928 14:45:28.388975 23585 raft_consensus.cc:504] T c8ca0a9372314a7a90e2c0fe30f86155 P b43bdfad8d3b4aa1bf499e9148871c0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ef635255234a8d8998cfa7951f1e32" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38363 } } peers { permanent_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45783 } } peers { permanent_uuid: "fa74e6c10d9241628165ee829514df21" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33867 } }
I20260928 14:45:28.389129 23585 leader_election.cc:288] T c8ca0a9372314a7a90e2c0fe30f86155 P b43bdfad8d3b4aa1bf499e9148871c0e [CANDIDATE]: Term 1 election: Requested vote from peers 90ef635255234a8d8998cfa7951f1e32 (127.9.33.66:38363), fa74e6c10d9241628165ee829514df21 (127.9.33.65:33867)
I20260928 14:45:28.389319 23380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8ca0a9372314a7a90e2c0fe30f86155" candidate_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90ef635255234a8d8998cfa7951f1e32"
I20260928 14:45:28.389401 23380 raft_consensus.cc:3037] T c8ca0a9372314a7a90e2c0fe30f86155 P 90ef635255234a8d8998cfa7951f1e32 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:45:28.389367 23246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8ca0a9372314a7a90e2c0fe30f86155" candidate_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa74e6c10d9241628165ee829514df21"
I20260928 14:45:28.389477 23246 raft_consensus.cc:3037] T c8ca0a9372314a7a90e2c0fe30f86155 P fa74e6c10d9241628165ee829514df21 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:45:28.390234 23380 raft_consensus.cc:2454] T c8ca0a9372314a7a90e2c0fe30f86155 P 90ef635255234a8d8998cfa7951f1e32 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b43bdfad8d3b4aa1bf499e9148871c0e in term 1.
I20260928 14:45:28.390256 23246 raft_consensus.cc:2454] T c8ca0a9372314a7a90e2c0fe30f86155 P fa74e6c10d9241628165ee829514df21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b43bdfad8d3b4aa1bf499e9148871c0e in term 1.
I20260928 14:45:28.390437 23447 leader_election.cc:302] T c8ca0a9372314a7a90e2c0fe30f86155 P b43bdfad8d3b4aa1bf499e9148871c0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90ef635255234a8d8998cfa7951f1e32, b43bdfad8d3b4aa1bf499e9148871c0e; no voters:
I20260928 14:45:28.390543 23585 raft_consensus.cc:2781] T c8ca0a9372314a7a90e2c0fe30f86155 P b43bdfad8d3b4aa1bf499e9148871c0e [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:45:28.390697 23585 raft_consensus.cc:686] T c8ca0a9372314a7a90e2c0fe30f86155 P b43bdfad8d3b4aa1bf499e9148871c0e [term 1 LEADER]: Becoming Leader. State: Replica: b43bdfad8d3b4aa1bf499e9148871c0e, State: Running, Role: LEADER
I20260928 14:45:28.390830 23585 consensus_queue.cc:237] T c8ca0a9372314a7a90e2c0fe30f86155 P b43bdfad8d3b4aa1bf499e9148871c0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ef635255234a8d8998cfa7951f1e32" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38363 } } peers { permanent_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45783 } } peers { permanent_uuid: "fa74e6c10d9241628165ee829514df21" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33867 } }
I20260928 14:45:28.391985 23100 catalog_manager.cc:5495] T c8ca0a9372314a7a90e2c0fe30f86155 P b43bdfad8d3b4aa1bf499e9148871c0e reported cstate change: term changed from 0 to 1, leader changed from <none> to b43bdfad8d3b4aa1bf499e9148871c0e (127.9.33.67). New cstate: current_term: 1 leader_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ef635255234a8d8998cfa7951f1e32" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa74e6c10d9241628165ee829514df21" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33867 } health_report { overall_health: UNKNOWN } } }
I20260928 14:45:28.400378 23574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:28.397603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35878 (local address 127.9.33.67:45783)
0928 14:45:28.397670 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:28.397673 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:28.397761 (+ 88us) server_negotiation.cc:407] Connection header received
0928 14:45:28.397812 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:28.397813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:28.397834 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:28.397897 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:45:28.398039 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.398523 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.399362 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.399667 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.399781 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:28.399974 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:45:28.400109 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:45:28.400197 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:28.400224 (+ 27us) server_negotiation.cc:299] Negotiation successful
0928 14:45:28.400252 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260928 14:45:28.402626 23380 raft_consensus.cc:1261] T c8ca0a9372314a7a90e2c0fe30f86155 P 90ef635255234a8d8998cfa7951f1e32 [term 1 FOLLOWER]: Refusing update from remote peer b43bdfad8d3b4aa1bf499e9148871c0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:45:28.402631 23246 raft_consensus.cc:1261] T c8ca0a9372314a7a90e2c0fe30f86155 P fa74e6c10d9241628165ee829514df21 [term 1 FOLLOWER]: Refusing update from remote peer b43bdfad8d3b4aa1bf499e9148871c0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:45:28.403024 23585 consensus_queue.cc:1035] T c8ca0a9372314a7a90e2c0fe30f86155 P b43bdfad8d3b4aa1bf499e9148871c0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "90ef635255234a8d8998cfa7951f1e32" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:45:28.403214 23587 consensus_queue.cc:1035] T c8ca0a9372314a7a90e2c0fe30f86155 P b43bdfad8d3b4aa1bf499e9148871c0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa74e6c10d9241628165ee829514df21" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:45:28.406220 23592 mvcc.cc:205] Tried to move back new op lower bound from 7334325159534190592 to 7334325159490252800. Current Snapshot: MvccSnapshot[applied={T|T < 7334325159534190592 or (T in {7334325159534190592})}]
I20260928 14:45:28.406566 23591 mvcc.cc:205] Tried to move back new op lower bound from 7334325159534190592 to 7334325159490252800. Current Snapshot: MvccSnapshot[applied={T|T < 7334325159534190592}]
I20260928 14:45:28.409077 23101 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:45:28.409202 23101 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:45:28.410241 23101 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043: Sending DeleteTablet for 3 replicas of tablet c8ca0a9372314a7a90e2c0fe30f86155
I20260928 14:45:28.410693 23360 tablet_service.cc:1514] Processing DeleteTablet for tablet c8ca0a9372314a7a90e2c0fe30f86155 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:45:28 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:32994
I20260928 14:45:28.410739 23226 tablet_service.cc:1514] Processing DeleteTablet for tablet c8ca0a9372314a7a90e2c0fe30f86155 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:45:28 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:49324
I20260928 14:45:28.410873 23494 tablet_service.cc:1514] Processing DeleteTablet for tablet c8ca0a9372314a7a90e2c0fe30f86155 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:45:28 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:35862
I20260928 14:45:28.411027 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 23163
I20260928 14:45:28.412460 23601 tablet_replica.cc:330] T c8ca0a9372314a7a90e2c0fe30f86155 P b43bdfad8d3b4aa1bf499e9148871c0e: stopping tablet replica
I20260928 14:45:28.412614 23601 raft_consensus.cc:2229] T c8ca0a9372314a7a90e2c0fe30f86155 P b43bdfad8d3b4aa1bf499e9148871c0e [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:45:28.412770 23601 raft_consensus.cc:2258] T c8ca0a9372314a7a90e2c0fe30f86155 P b43bdfad8d3b4aa1bf499e9148871c0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:45:28.413260 23601 ts_tablet_manager.cc:1905] T c8ca0a9372314a7a90e2c0fe30f86155 P b43bdfad8d3b4aa1bf499e9148871c0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:45:28.414983 23601 ts_tablet_manager.cc:1918] T c8ca0a9372314a7a90e2c0fe30f86155 P b43bdfad8d3b4aa1bf499e9148871c0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:45:28.415083 23601 log.cc:1196] T c8ca0a9372314a7a90e2c0fe30f86155 P b43bdfad8d3b4aa1bf499e9148871c0e: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/wal/wals/c8ca0a9372314a7a90e2c0fe30f86155
I20260928 14:45:28.415380 23601 ts_tablet_manager.cc:1939] T c8ca0a9372314a7a90e2c0fe30f86155 P b43bdfad8d3b4aa1bf499e9148871c0e: Deleting consensus metadata
I20260928 14:45:28.415859 23599 tablet_replica.cc:330] T c8ca0a9372314a7a90e2c0fe30f86155 P 90ef635255234a8d8998cfa7951f1e32: stopping tablet replica
I20260928 14:45:28.415884 23087 catalog_manager.cc:4841] TS b43bdfad8d3b4aa1bf499e9148871c0e (127.9.33.67:45783): tablet c8ca0a9372314a7a90e2c0fe30f86155 (table test-table [id=141e4fd4dc9a44bf8f43852e002d1402]) successfully deleted
I20260928 14:45:28.416007 23599 raft_consensus.cc:2229] T c8ca0a9372314a7a90e2c0fe30f86155 P 90ef635255234a8d8998cfa7951f1e32 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:45:28.416144 23599 raft_consensus.cc:2258] T c8ca0a9372314a7a90e2c0fe30f86155 P 90ef635255234a8d8998cfa7951f1e32 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260928 14:45:28.418272 23086 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.33.65:33867 (error 108)
I20260928 14:45:28.419139 23599 ts_tablet_manager.cc:1905] T c8ca0a9372314a7a90e2c0fe30f86155 P 90ef635255234a8d8998cfa7951f1e32: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:45:28.419304 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 23297
I20260928 14:45:28.419482 23573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:28.419338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:33867 (local address 127.0.0.1:49328)
0928 14:45:28.419400 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:28.419444 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.65:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260928 14:45:28.419602 23086 catalog_manager.cc:4584] TS fa74e6c10d9241628165ee829514df21 (127.9.33.65:33867): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c8ca0a9372314a7a90e2c0fe30f86155: Network error: Client connection negotiation failed: client connection to 127.9.33.65:33867: connect: Connection refused (error 111)
I20260928 14:45:28.427019 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 23431
I20260928 14:45:28.427174 23573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:28.427005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:38363 (local address 127.0.0.1:33000)
0928 14:45:28.427094 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:28.427132 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.66:38363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260928 14:45:28.432170 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 23069
I20260928 14:45:28.437949 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:46763
--webserver_interface=127.9.33.126
--webserver_port=46303
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:35485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:46763
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:45:28.522320 23603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:45:28.522562 23603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:45:28.522588 23603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:45:28.523882 23603 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:45:28.523931 23603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:45:28.523945 23603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:45:28.523958 23603 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:45:28.523970 23603 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:45:28.525683 23603 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:35485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:46763
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:46763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=46303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:45:28.525929 23603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:45:28.526113 23603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:45:28.535331 23609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:28.535331 23612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:28.535331 23610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:45:28.535415 23603 server_base.cc:1034] running on GCE node
I20260928 14:45:28.536051 23603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:45:28.536455 23603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:45:28.537660 23603 hybrid_clock.cc:648] HybridClock initialized: now 1790606728537639 us; error 38 us; skew 500 ppm
Sep 28 14:45:28 dist-test-slave-pv0w krb5kdc[23056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606728, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:45:28.540401 23603 init.cc:377] Logged in from keytab as kudu/127.9.33.126@KRBTEST.COM (short username kudu)
I20260928 14:45:28.540867 23603 webserver.cc:460] Webserver started at http://127.9.33.126:46303/ using document root <none> and password file <none>
I20260928 14:45:28.541083 23603 fs_manager.cc:359] Metadata directory not provided
I20260928 14:45:28.541137 23603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:45:28.542604 23603 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:45:28.543293 23619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:28.543519 23603 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:45:28.543619 23603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "ad4350e8668941ea9fa505f99e327043"
format_stamp: "Formatted at 2026-09-28 14:45:27 on dist-test-slave-pv0w"
server_key: "1e3c3ed5b194beb1c194636d9803ebf3"
server_key_iv: "3fc69158bdc24fcfd5336afb59720978"
server_key_version: "encryptionkey@0"
I20260928 14:45:28.544080 23603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:45:28.564225 23603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:45:28.566597 23603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:45:28.566721 23603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:45:28.570840 23603 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:46763
I20260928 14:45:28.570945 23671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:46763 every 8 connection(s)
I20260928 14:45:28.571357 23603 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:45:28.573148 23672 sys_catalog.cc:263] Verifying existing consensus state
I20260928 14:45:28.573792 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 23603
I20260928 14:45:28.573918 23672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043: Bootstrap starting.
I20260928 14:45:28.574131 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.65:33867
--local_ip_for_outbound_sockets=127.9.33.65
--tserver_master_addrs=127.9.33.126:46763
--webserver_port=46089
--webserver_interface=127.9.33.65
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:35485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:45:28.582739 23672 log.cc:824] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043: Log is configured to *not* fsync() on all Append() calls
I20260928 14:45:28.584937 23672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 14:45:28.585268 23672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043: Bootstrap complete.
I20260928 14:45:28.587705 23672 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad4350e8668941ea9fa505f99e327043" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 46763 } }
I20260928 14:45:28.588068 23672 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad4350e8668941ea9fa505f99e327043, State: Initialized, Role: FOLLOWER
I20260928 14:45:28.588259 23672 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad4350e8668941ea9fa505f99e327043" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 46763 } }
I20260928 14:45:28.588338 23672 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:45:28.588397 23672 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:45:28.588456 23672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [term 1 FOLLOWER]: Advancing to term 2
I20260928 14:45:28.589597 23672 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad4350e8668941ea9fa505f99e327043" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 46763 } }
I20260928 14:45:28.589732 23672 leader_election.cc:302] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad4350e8668941ea9fa505f99e327043; no voters:
I20260928 14:45:28.589969 23672 leader_election.cc:288] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260928 14:45:28.590014 23676 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [term 2 FOLLOWER]: Leader election won for term 2
I20260928 14:45:28.590147 23676 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [term 2 LEADER]: Becoming Leader. State: Replica: ad4350e8668941ea9fa505f99e327043, State: Running, Role: LEADER
I20260928 14:45:28.590243 23676 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad4350e8668941ea9fa505f99e327043" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 46763 } }
I20260928 14:45:28.590265 23672 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:45:28.590525 23676 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad4350e8668941ea9fa505f99e327043. Latest consensus state: current_term: 2 leader_uuid: "ad4350e8668941ea9fa505f99e327043" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad4350e8668941ea9fa505f99e327043" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 46763 } } }
I20260928 14:45:28.590605 23676 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [sys.catalog]: This master's current role is: LEADER
I20260928 14:45:28.590533 23678 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ad4350e8668941ea9fa505f99e327043" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad4350e8668941ea9fa505f99e327043" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 46763 } } }
I20260928 14:45:28.590808 23678 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [sys.catalog]: This master's current role is: LEADER
I20260928 14:45:28.590824 23680 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:45:28.591827 23680 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:45:28.592103 23680 catalog_manager.cc:1259] Loaded cluster ID: d38f0742c7664947ac2e98492bc58dbd
I20260928 14:45:28.592257 23680 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:45:28.592576 23680 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260928 14:45:28.597307 23680 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260928 14:45:28.600214 23680 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:45:28.600483 23680 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043: Loaded TSK: 0
I20260928 14:45:28.600773 23680 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 14:45:28.663055 23674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:45:28.663331 23674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:45:28.663375 23674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:45:28.664716 23674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:45:28.664796 23674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:45:28.664844 23674 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.65
I20260928 14:45:28.666452 23674 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:35485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.65:33867
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.33.65
--webserver_port=46089
--tserver_master_addrs=127.9.33.126:46763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.65
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:45:28.666754 23674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:45:28.667004 23674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:45:28.672523 23699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:28.672634 23702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:45:28.672530 23674 server_base.cc:1034] running on GCE node
W20260928 14:45:28.672725 23700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:45:28.673111 23674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:45:28.673439 23674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:45:28.674635 23674 hybrid_clock.cc:648] HybridClock initialized: now 1790606728674617 us; error 27 us; skew 500 ppm
Sep 28 14:45:28 dist-test-slave-pv0w krb5kdc[23056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606728, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:45:28.677929 23674 init.cc:377] Logged in from keytab as kudu/127.9.33.65@KRBTEST.COM (short username kudu)
I20260928 14:45:28.678524 23674 webserver.cc:460] Webserver started at http://127.9.33.65:46089/ using document root <none> and password file <none>
I20260928 14:45:28.678807 23674 fs_manager.cc:359] Metadata directory not provided
I20260928 14:45:28.678943 23674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:45:28.680932 23674 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:45:28.681823 23709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:28.682085 23674 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:45:28.682235 23674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/wal
uuid: "fa74e6c10d9241628165ee829514df21"
format_stamp: "Formatted at 2026-09-28 14:45:27 on dist-test-slave-pv0w"
server_key: "6d2358cd2f05abf78e5a16810320fbb7"
server_key_iv: "f5cb5c9ee5ac786dda0ec001022df31f"
server_key_version: "encryptionkey@0"
I20260928 14:45:28.682631 23674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:45:28.696393 23674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:45:28.698849 23674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:45:28.699030 23674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:45:28.699297 23674 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:45:28.699892 23716 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260928 14:45:28.702020 23674 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260928 14:45:28.702102 23674 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260928 14:45:28.702155 23674 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260928 14:45:28.702914 23674 ts_tablet_manager.cc:610] Registered 1 tablets
I20260928 14:45:28.702975 23674 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260928 14:45:28.703063 23716 tablet_bootstrap.cc:492] T c8ca0a9372314a7a90e2c0fe30f86155 P fa74e6c10d9241628165ee829514df21: Bootstrap starting.
I20260928 14:45:28.710522 23674 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.65:33867
I20260928 14:45:28.710561 23823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.65:33867 every 8 connection(s)
I20260928 14:45:28.710979 23674 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
I20260928 14:45:28.714534 23716 log.cc:824] T c8ca0a9372314a7a90e2c0fe30f86155 P fa74e6c10d9241628165ee829514df21: Log is configured to *not* fsync() on all Append() calls
I20260928 14:45:28.717655 23716 tablet_bootstrap.cc:492] T c8ca0a9372314a7a90e2c0fe30f86155 P fa74e6c10d9241628165ee829514df21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 14:45:28.718025 23716 tablet_bootstrap.cc:492] T c8ca0a9372314a7a90e2c0fe30f86155 P fa74e6c10d9241628165ee829514df21: Bootstrap complete.
I20260928 14:45:28.719306 23716 ts_tablet_manager.cc:1397] T c8ca0a9372314a7a90e2c0fe30f86155 P fa74e6c10d9241628165ee829514df21: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.009s
I20260928 14:45:28.719947 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 23674
I20260928 14:45:28.720264 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.66:38363
--local_ip_for_outbound_sockets=127.9.33.66
--tserver_master_addrs=127.9.33.126:46763
--webserver_port=44581
--webserver_interface=127.9.33.66
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:35485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:45:28.720860 23716 raft_consensus.cc:348] T c8ca0a9372314a7a90e2c0fe30f86155 P fa74e6c10d9241628165ee829514df21 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ef635255234a8d8998cfa7951f1e32" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38363 } } peers { permanent_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45783 } } peers { permanent_uuid: "fa74e6c10d9241628165ee829514df21" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33867 } }
I20260928 14:45:28.721196 23716 raft_consensus.cc:729] T c8ca0a9372314a7a90e2c0fe30f86155 P fa74e6c10d9241628165ee829514df21 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa74e6c10d9241628165ee829514df21, State: Initialized, Role: FOLLOWER
I20260928 14:45:28.721369 23716 consensus_queue.cc:260] T c8ca0a9372314a7a90e2c0fe30f86155 P fa74e6c10d9241628165ee829514df21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ef635255234a8d8998cfa7951f1e32" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38363 } } peers { permanent_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45783 } } peers { permanent_uuid: "fa74e6c10d9241628165ee829514df21" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33867 } }
I20260928 14:45:28.721668 23716 ts_tablet_manager.cc:1428] T c8ca0a9372314a7a90e2c0fe30f86155 P fa74e6c10d9241628165ee829514df21: Time spent starting tablet: real 0.002s user 0.006s sys 0.000s
Sep 28 14:45:28 dist-test-slave-pv0w krb5kdc[23056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606728, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:45:28.726454 23827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:28.711818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:39835 (local address 127.9.33.126:46763)
0928 14:45:28.714974 (+ 3156us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:28.714987 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:28.715042 (+ 55us) server_negotiation.cc:407] Connection header received
0928 14:45:28.715218 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:28.715226 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:28.715307 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:28.715506 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:28.716351 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.717299 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.718535 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.718998 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.719139 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:28.722480 (+ 3341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:28.722485 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:28.722486 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:28.722500 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:28.724358 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:28.725019 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:28.725027 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:28.725028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:28.725120 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:28.725410 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:28.725412 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:28.725412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:28.725661 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:28.725771 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:28.726083 (+ 312us) server_negotiation.cc:299] Negotiation successful
0928 14:45:28.726221 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3086,"thread_start_us":110,"threads_started":1}
I20260928 14:45:28.726454 23826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:28.711765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.65:39835)
0928 14:45:28.711971 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:28.712018 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:28.712514 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:28.715534 (+ 3020us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:28.715539 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:28.715960 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:45:28.716165 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:28.716214 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.717425 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.717426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:28.718437 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:28.718440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.719119 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.719120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:28.719452 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:28.720213 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:45:28.720234 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:45:28.722336 (+ 2102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:45:28.724563 (+ 2227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:28.724571 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:28.724573 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:28.724936 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:28.725201 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:28.725201 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:28.725202 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:28.725335 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:28.725738 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:45:28.725745 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:45:28.725910 (+ 165us) client_negotiation.cc:769] Sending connection context
0928 14:45:28.726016 (+ 106us) client_negotiation.cc:240] Negotiation successful
0928 14:45:28.726221 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":123,"threads_started":1}
I20260928 14:45:28.727645 23824 heartbeater.cc:347] Connected to a master server at 127.9.33.126:46763
I20260928 14:45:28.727792 23824 heartbeater.cc:464] Registering TS with master...
I20260928 14:45:28.727993 23824 heartbeater.cc:511] Master 127.9.33.126:46763 requested a full tablet report, sending...
I20260928 14:45:28.728765 23636 ts_manager.cc:194] Registered new tserver with Master: fa74e6c10d9241628165ee829514df21 (127.9.33.65:33867)
I20260928 14:45:28.730038 23636 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.65@KRBTEST.COM'} at 127.9.33.65:39835
I20260928 14:45:28.731009 23824 heartbeater.cc:503] Master 127.9.33.126:46763 was elected leader, sending a full tablet report...
Sep 28 14:45:28 dist-test-slave-pv0w krb5kdc[23056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606728, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for kudu/127.9.33.65@KRBTEST.COM
I20260928 14:45:28.739339 23831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:28.729382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:33867 (local address 127.0.0.1:49340)
0928 14:45:28.729876 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:28.729915 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:28.729981 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:28.730484 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:28.730488 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:28.730853 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:45:28.730950 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:28.730953 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.732417 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.732418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:28.733482 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:28.733485 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.734022 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.734023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:28.734135 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:28.734712 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:45:28.734716 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:45:28.736100 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:45:28.738043 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:28.738052 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:28.738054 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:28.738281 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:28.738528 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:28.738530 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:28.738530 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:28.738601 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:28.738911 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:45:28.738918 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:45:28.738977 (+ 59us) client_negotiation.cc:769] Sending connection context
0928 14:45:28.739073 (+ 96us) client_negotiation.cc:240] Negotiation successful
0928 14:45:28.739218 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":100,"threads_started":1}
I20260928 14:45:28.739356 23832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:28.729644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49340 (local address 127.9.33.65:33867)
0928 14:45:28.730192 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:28.730197 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:28.730207 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:45:28.730246 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:28.730248 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:28.730273 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:28.730420 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:28.731043 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.732324 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.733594 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.733937 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.734022 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:28.736199 (+ 2177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:28.736209 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:28.736209 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:28.736225 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:28.737931 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:28.738371 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:28.738374 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:28.738375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:28.738450 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:28.738665 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:28.738666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:28.738666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:28.738833 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:28.738924 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:28.739135 (+ 211us) server_negotiation.cc:299] Negotiation successful
0928 14:45:28.739229 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":130,"threads_started":1}
I20260928 14:45:28.740190 23758 tablet_service.cc:1514] Processing DeleteTablet for tablet c8ca0a9372314a7a90e2c0fe30f86155 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:45:28 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:49340
I20260928 14:45:28.740815 23833 tablet_replica.cc:330] stopping tablet replica
I20260928 14:45:28.740952 23833 raft_consensus.cc:2229] T c8ca0a9372314a7a90e2c0fe30f86155 P fa74e6c10d9241628165ee829514df21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:45:28.741029 23833 raft_consensus.cc:2258] T c8ca0a9372314a7a90e2c0fe30f86155 P fa74e6c10d9241628165ee829514df21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:45:28.741355 23833 ts_tablet_manager.cc:1905] T c8ca0a9372314a7a90e2c0fe30f86155 P fa74e6c10d9241628165ee829514df21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:45:28.743607 23833 ts_tablet_manager.cc:1918] T c8ca0a9372314a7a90e2c0fe30f86155 P fa74e6c10d9241628165ee829514df21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:45:28.743810 23833 log.cc:1196] T c8ca0a9372314a7a90e2c0fe30f86155 P fa74e6c10d9241628165ee829514df21: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/wal/wals/c8ca0a9372314a7a90e2c0fe30f86155
I20260928 14:45:28.744071 23833 ts_tablet_manager.cc:1939] T c8ca0a9372314a7a90e2c0fe30f86155 P fa74e6c10d9241628165ee829514df21: Deleting consensus metadata
I20260928 14:45:28.744752 23621 catalog_manager.cc:4841] TS fa74e6c10d9241628165ee829514df21 (127.9.33.65:33867): tablet c8ca0a9372314a7a90e2c0fe30f86155 (table test-table [id=141e4fd4dc9a44bf8f43852e002d1402]) successfully deleted
W20260928 14:45:28.814944 23830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:45:28.815214 23830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:45:28.815245 23830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:45:28.816548 23830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:45:28.816598 23830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:45:28.816632 23830 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.66
I20260928 14:45:28.818286 23830 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:35485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.66:38363
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.33.66
--webserver_port=44581
--tserver_master_addrs=127.9.33.126:46763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.66
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:45:28.818511 23830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:45:28.818707 23830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 14:45:28.824417 23830 server_base.cc:1034] running on GCE node
W20260928 14:45:28.824393 23841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:28.824379 23838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:28.824388 23839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:45:28.824833 23830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:45:28.825157 23830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:45:28.826375 23830 hybrid_clock.cc:648] HybridClock initialized: now 1790606728826348 us; error 41 us; skew 500 ppm
Sep 28 14:45:28 dist-test-slave-pv0w krb5kdc[23056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606728, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:45:28.829469 23830 init.cc:377] Logged in from keytab as kudu/127.9.33.66@KRBTEST.COM (short username kudu)
I20260928 14:45:28.829991 23830 webserver.cc:460] Webserver started at http://127.9.33.66:44581/ using document root <none> and password file <none>
I20260928 14:45:28.830251 23830 fs_manager.cc:359] Metadata directory not provided
I20260928 14:45:28.830340 23830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:45:28.831930 23830 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260928 14:45:28.832845 23848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:28.833079 23830 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:45:28.833177 23830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/wal
uuid: "90ef635255234a8d8998cfa7951f1e32"
format_stamp: "Formatted at 2026-09-28 14:45:28 on dist-test-slave-pv0w"
server_key: "8ccbe6dc6913e0350f1ccc4933697848"
server_key_iv: "5ed48d52c22b6c84de3afc350ee33f4c"
server_key_version: "encryptionkey@0"
I20260928 14:45:28.833600 23830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:45:28.858309 23830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:45:28.860983 23830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:45:28.861140 23830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:45:28.861466 23830 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:45:28.862023 23855 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260928 14:45:28.863593 23830 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260928 14:45:28.863668 23830 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260928 14:45:28.863749 23830 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260928 14:45:28.864490 23830 ts_tablet_manager.cc:610] Registered 1 tablets
I20260928 14:45:28.864548 23830 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260928 14:45:28.864646 23855 tablet_bootstrap.cc:492] T c8ca0a9372314a7a90e2c0fe30f86155 P 90ef635255234a8d8998cfa7951f1e32: Bootstrap starting.
I20260928 14:45:28.866294 23855 log.cc:824] T c8ca0a9372314a7a90e2c0fe30f86155 P 90ef635255234a8d8998cfa7951f1e32: Log is configured to *not* fsync() on all Append() calls
I20260928 14:45:28.869350 23855 tablet_bootstrap.cc:492] T c8ca0a9372314a7a90e2c0fe30f86155 P 90ef635255234a8d8998cfa7951f1e32: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 14:45:28.869664 23855 tablet_bootstrap.cc:492] T c8ca0a9372314a7a90e2c0fe30f86155 P 90ef635255234a8d8998cfa7951f1e32: Bootstrap complete.
I20260928 14:45:28.869853 23855 ts_tablet_manager.cc:1397] T c8ca0a9372314a7a90e2c0fe30f86155 P 90ef635255234a8d8998cfa7951f1e32: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260928 14:45:28.871315 23855 raft_consensus.cc:348] T c8ca0a9372314a7a90e2c0fe30f86155 P 90ef635255234a8d8998cfa7951f1e32 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ef635255234a8d8998cfa7951f1e32" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38363 } } peers { permanent_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45783 } } peers { permanent_uuid: "fa74e6c10d9241628165ee829514df21" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33867 } }
I20260928 14:45:28.871594 23855 raft_consensus.cc:729] T c8ca0a9372314a7a90e2c0fe30f86155 P 90ef635255234a8d8998cfa7951f1e32 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90ef635255234a8d8998cfa7951f1e32, State: Initialized, Role: FOLLOWER
I20260928 14:45:28.871712 23830 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.66:38363
I20260928 14:45:28.871791 23855 consensus_queue.cc:260] T c8ca0a9372314a7a90e2c0fe30f86155 P 90ef635255234a8d8998cfa7951f1e32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90ef635255234a8d8998cfa7951f1e32" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38363 } } peers { permanent_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45783 } } peers { permanent_uuid: "fa74e6c10d9241628165ee829514df21" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33867 } }
I20260928 14:45:28.871868 23963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.66:38363 every 8 connection(s)
I20260928 14:45:28.872112 23855 ts_tablet_manager.cc:1428] T c8ca0a9372314a7a90e2c0fe30f86155 P 90ef635255234a8d8998cfa7951f1e32: Time spent starting tablet: real 0.002s user 0.000s sys 0.004s
I20260928 14:45:28.872151 23830 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
I20260928 14:45:28.877241 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 23830
I20260928 14:45:28.877607 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.67:45783
--local_ip_for_outbound_sockets=127.9.33.67
--tserver_master_addrs=127.9.33.126:46763
--webserver_port=46637
--webserver_interface=127.9.33.67
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:35485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 14:45:28 dist-test-slave-pv0w krb5kdc[23056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606728, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:45:28.882669 23827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:28.872915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:55447 (local address 127.9.33.126:46763)
0928 14:45:28.872985 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:28.872988 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:28.873194 (+ 206us) server_negotiation.cc:407] Connection header received
0928 14:45:28.873390 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:28.873391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:28.873410 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:28.873457 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:28.874125 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.874664 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.875784 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.876016 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.876104 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:28.879124 (+ 3020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:28.879127 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:28.879127 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:28.879136 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:28.880790 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:28.881400 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:28.881402 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:28.881403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:28.881468 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:28.881844 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:28.881845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:28.881846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:28.881988 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:28.882059 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:28.882494 (+ 435us) server_negotiation.cc:299] Negotiation successful
0928 14:45:28.882593 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260928 14:45:28.883019 23967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:28.872925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.66:55447)
0928 14:45:28.873075 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:28.873108 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:28.873289 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:28.873520 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:28.873524 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:28.873809 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:45:28.874018 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:28.874023 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.874795 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.874797 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:28.875644 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:28.875646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.876119 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.876120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:28.876265 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:28.876841 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:45:28.876849 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:45:28.878965 (+ 2116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:45:28.880902 (+ 1937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:28.880913 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:28.880916 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:28.881306 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:28.881551 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:28.881552 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:28.881553 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:28.881734 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:28.882071 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:45:28.882081 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:45:28.882311 (+ 230us) client_negotiation.cc:769] Sending connection context
0928 14:45:28.882454 (+ 143us) client_negotiation.cc:240] Negotiation successful
0928 14:45:28.882697 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":63,"threads_started":1}
I20260928 14:45:28.883781 23965 heartbeater.cc:347] Connected to a master server at 127.9.33.126:46763
I20260928 14:45:28.883920 23965 heartbeater.cc:464] Registering TS with master...
I20260928 14:45:28.884145 23965 heartbeater.cc:511] Master 127.9.33.126:46763 requested a full tablet report, sending...
I20260928 14:45:28.884859 23636 ts_manager.cc:194] Registered new tserver with Master: 90ef635255234a8d8998cfa7951f1e32 (127.9.33.66:38363)
I20260928 14:45:28.885496 23636 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.66@KRBTEST.COM'} at 127.9.33.66:55447
Sep 28 14:45:28 dist-test-slave-pv0w krb5kdc[23056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606728, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for kudu/127.9.33.66@KRBTEST.COM
I20260928 14:45:28.894667 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:28.885691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33008 (local address 127.9.33.66:38363)
0928 14:45:28.885910 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:28.885914 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:28.885925 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:45:28.885963 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:28.885966 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:28.885983 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:28.886102 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:28.886506 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.887181 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.888387 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.888728 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.888799 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:28.891280 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:28.891294 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:28.891294 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:28.891303 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:28.893241 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:28.893656 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:28.893660 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:28.893662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:28.893725 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:28.893957 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:28.893958 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:28.893958 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:28.894180 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:28.894265 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:28.894448 (+ 183us) server_negotiation.cc:299] Negotiation successful
0928 14:45:28.894550 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":148,"threads_started":1}
I20260928 14:45:28.894655 23831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:28.885277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:38363 (local address 127.0.0.1:33008)
0928 14:45:28.885341 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:28.885352 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:28.885396 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:28.886119 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:28.886120 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:28.886349 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:45:28.886411 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:28.886415 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.887276 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.887277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:28.888282 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:28.888290 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:28.888815 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:28.888816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:28.888913 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:28.889448 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:45:28.889454 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:45:28.891168 (+ 1714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:45:28.893353 (+ 2185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:28.893354 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:28.893355 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:28.893564 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:28.893817 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:28.893818 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:28.893818 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:28.893890 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:28.894252 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:45:28.894253 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:45:28.894318 (+ 65us) client_negotiation.cc:769] Sending connection context
0928 14:45:28.894408 (+ 90us) client_negotiation.cc:240] Negotiation successful
0928 14:45:28.894545 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260928 14:45:28.895197 23898 tablet_service.cc:1514] Processing DeleteTablet for tablet c8ca0a9372314a7a90e2c0fe30f86155 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:45:28 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:33008
I20260928 14:45:28.895936 23970 tablet_replica.cc:330] stopping tablet replica
I20260928 14:45:28.896081 23970 raft_consensus.cc:2229] T c8ca0a9372314a7a90e2c0fe30f86155 P 90ef635255234a8d8998cfa7951f1e32 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:45:28.896173 23970 raft_consensus.cc:2258] T c8ca0a9372314a7a90e2c0fe30f86155 P 90ef635255234a8d8998cfa7951f1e32 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:45:28.896507 23970 ts_tablet_manager.cc:1905] T c8ca0a9372314a7a90e2c0fe30f86155 P 90ef635255234a8d8998cfa7951f1e32: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:45:28.898883 23970 ts_tablet_manager.cc:1918] T c8ca0a9372314a7a90e2c0fe30f86155 P 90ef635255234a8d8998cfa7951f1e32: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:45:28.898988 23970 log.cc:1196] T c8ca0a9372314a7a90e2c0fe30f86155 P 90ef635255234a8d8998cfa7951f1e32: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/wal/wals/c8ca0a9372314a7a90e2c0fe30f86155
I20260928 14:45:28.899225 23970 ts_tablet_manager.cc:1939] T c8ca0a9372314a7a90e2c0fe30f86155 P 90ef635255234a8d8998cfa7951f1e32: Deleting consensus metadata
I20260928 14:45:28.899881 23620 catalog_manager.cc:4841] TS 90ef635255234a8d8998cfa7951f1e32 (127.9.33.66:38363): tablet c8ca0a9372314a7a90e2c0fe30f86155 (table test-table [id=141e4fd4dc9a44bf8f43852e002d1402]) successfully deleted
W20260928 14:45:28.965559 23968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:45:28.965780 23968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:45:28.965806 23968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:45:28.966995 23968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:45:28.967043 23968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:45:28.967070 23968 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.67
I20260928 14:45:28.968580 23968 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:35485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.67:45783
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.33.67
--webserver_port=46637
--tserver_master_addrs=127.9.33.126:46763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.67
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:45:28.968853 23968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:45:28.969103 23968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:45:28.974668 23975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:28.974689 23976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:28.974682 23978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:45:28.974896 23968 server_base.cc:1034] running on GCE node
I20260928 14:45:28.975107 23968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:45:28.975386 23968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:45:28.976557 23968 hybrid_clock.cc:648] HybridClock initialized: now 1790606728976539 us; error 43 us; skew 500 ppm
Sep 28 14:45:28 dist-test-slave-pv0w krb5kdc[23056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606728, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:45:28.979426 23968 init.cc:377] Logged in from keytab as kudu/127.9.33.67@KRBTEST.COM (short username kudu)
I20260928 14:45:28.979926 23968 webserver.cc:460] Webserver started at http://127.9.33.67:46637/ using document root <none> and password file <none>
I20260928 14:45:28.980119 23968 fs_manager.cc:359] Metadata directory not provided
I20260928 14:45:28.980171 23968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:45:28.981519 23968 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:45:28.982105 23985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:28.982271 23968 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:45:28.982359 23968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/wal
uuid: "b43bdfad8d3b4aa1bf499e9148871c0e"
format_stamp: "Formatted at 2026-09-28 14:45:28 on dist-test-slave-pv0w"
server_key: "01e19ff0a4f3acc9e3040683e1657e1d"
server_key_iv: "e35e4b450a85b2eae000a3c8b2da56df"
server_key_version: "encryptionkey@0"
I20260928 14:45:28.982789 23968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:45:28.997548 23968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:45:28.999836 23968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:45:28.999976 23968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:45:29.000214 23968 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:45:29.000487 23968 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:45:29.000519 23968 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:29.000540 23968 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:45:29.000553 23968 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:29.006198 23968 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.67:45783
I20260928 14:45:29.006285 24098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.67:45783 every 8 connection(s)
I20260928 14:45:29.006646 23968 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
I20260928 14:45:29.013366 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 23968
Sep 28 14:45:29 dist-test-slave-pv0w krb5kdc[23056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606728, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:45:29.017776 23827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:29.007607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:37129 (local address 127.9.33.126:46763)
0928 14:45:29.007683 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:29.007687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:29.008037 (+ 350us) server_negotiation.cc:407] Connection header received
0928 14:45:29.008670 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:29.008671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:29.008683 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:29.008760 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:29.009812 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:29.010334 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:29.011426 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:29.011625 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:29.011691 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:29.014277 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:29.014283 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:29.014283 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:29.014291 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:29.016013 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:29.016495 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:29.016497 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:29.016498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:29.016564 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:29.016904 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:29.016906 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:29.016906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:29.017048 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:29.017127 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:29.017530 (+ 403us) server_negotiation.cc:299] Negotiation successful
0928 14:45:29.017638 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 14:45:29.017845 24101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:29.007657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.67:37129)
0928 14:45:29.007890 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:29.007944 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:29.008564 (+ 620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:29.008846 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:29.008850 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:29.009452 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:45:29.009673 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:29.009677 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:29.010452 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:29.010453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:29.011311 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:29.011313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:29.011772 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:29.011774 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:29.011911 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:29.012457 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:45:29.012466 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:45:29.014178 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:45:29.016119 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:29.016129 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:29.016131 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:29.016408 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:29.016645 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:29.016645 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:29.016646 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:29.016807 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:29.017129 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:45:29.017136 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:45:29.017343 (+ 207us) client_negotiation.cc:769] Sending connection context
0928 14:45:29.017468 (+ 125us) client_negotiation.cc:240] Negotiation successful
0928 14:45:29.017642 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":115,"threads_started":1}
I20260928 14:45:29.018751 24099 heartbeater.cc:347] Connected to a master server at 127.9.33.126:46763
I20260928 14:45:29.018893 24099 heartbeater.cc:464] Registering TS with master...
I20260928 14:45:29.019119 24099 heartbeater.cc:511] Master 127.9.33.126:46763 requested a full tablet report, sending...
I20260928 14:45:29.019577 23636 ts_manager.cc:194] Registered new tserver with Master: b43bdfad8d3b4aa1bf499e9148871c0e (127.9.33.67:45783)
I20260928 14:45:29.020043 23636 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.67@KRBTEST.COM'} at 127.9.33.67:37129
I20260928 14:45:29.021600 24103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:29.013780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41092 (local address 127.9.33.126:46763)
0928 14:45:29.013954 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:29.013957 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:29.014013 (+ 56us) server_negotiation.cc:407] Connection header received
0928 14:45:29.014058 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:29.014059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:29.014073 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:29.014150 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:29.014714 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:29.015463 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:29.016545 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:29.016850 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:29.016922 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:29.018569 (+ 1647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:29.018570 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:29.018571 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:29.018579 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:29.020293 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:29.020789 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:29.020790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:29.020791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:29.020854 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:29.021067 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:29.021068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:29.021069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:29.021187 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:29.021361 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:29.021401 (+ 40us) server_negotiation.cc:299] Negotiation successful
0928 14:45:29.021492 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":95,"threads_started":1}
I20260928 14:45:29.022431 9349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:45:29.025476 24103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:29.022843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41104 (local address 127.9.33.126:46763)
0928 14:45:29.022914 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:29.022916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:29.022983 (+ 67us) server_negotiation.cc:407] Connection header received
0928 14:45:29.023063 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:29.023063 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:29.023100 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:29.023192 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:45:29.023359 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:29.023902 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:29.024676 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:29.024894 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:29.024974 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:29.025145 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:45:29.025244 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:45:29.025350 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:29.025367 (+ 17us) server_negotiation.cc:299] Negotiation successful
0928 14:45:29.025392 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260928 14:45:29.025895 23636 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:41104:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:45:29.027611 23636 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:45:29.030100 23898 tablet_service.cc:1467] Processing CreateTablet for tablet 7c7d62a1e9d64474a3a898bd887bf3b3 (DEFAULT_TABLE table=test-table [id=5aab3e0c3202443682c363221c017124]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:45:29.030081 23758 tablet_service.cc:1467] Processing CreateTablet for tablet 7c7d62a1e9d64474a3a898bd887bf3b3 (DEFAULT_TABLE table=test-table [id=5aab3e0c3202443682c363221c017124]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:45:29.030316 23758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c7d62a1e9d64474a3a898bd887bf3b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:45:29.030325 23898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c7d62a1e9d64474a3a898bd887bf3b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:45:29.031863 23716 tablet_bootstrap.cc:492] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21: Bootstrap starting.
I20260928 14:45:29.031863 23855 tablet_bootstrap.cc:492] T 7c7d62a1e9d64474a3a898bd887bf3b3 P 90ef635255234a8d8998cfa7951f1e32: Bootstrap starting.
I20260928 14:45:29.032346 23855 tablet_bootstrap.cc:654] T 7c7d62a1e9d64474a3a898bd887bf3b3 P 90ef635255234a8d8998cfa7951f1e32: Neither blocks nor log segments found. Creating new log.
I20260928 14:45:29.032348 23716 tablet_bootstrap.cc:654] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21: Neither blocks nor log segments found. Creating new log.
I20260928 14:45:29.033352 23855 tablet_bootstrap.cc:492] T 7c7d62a1e9d64474a3a898bd887bf3b3 P 90ef635255234a8d8998cfa7951f1e32: No bootstrap required, opened a new log
I20260928 14:45:29.033420 23855 ts_tablet_manager.cc:1397] T 7c7d62a1e9d64474a3a898bd887bf3b3 P 90ef635255234a8d8998cfa7951f1e32: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260928 14:45:29.033550 23716 tablet_bootstrap.cc:492] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21: No bootstrap required, opened a new log
I20260928 14:45:29.033586 23716 ts_tablet_manager.cc:1397] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:45:29.033564 23855 raft_consensus.cc:348] T 7c7d62a1e9d64474a3a898bd887bf3b3 P 90ef635255234a8d8998cfa7951f1e32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45783 } } peers { permanent_uuid: "fa74e6c10d9241628165ee829514df21" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33867 } } peers { permanent_uuid: "90ef635255234a8d8998cfa7951f1e32" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38363 } }
I20260928 14:45:29.033648 23855 raft_consensus.cc:374] T 7c7d62a1e9d64474a3a898bd887bf3b3 P 90ef635255234a8d8998cfa7951f1e32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:45:29.033708 23855 raft_consensus.cc:729] T 7c7d62a1e9d64474a3a898bd887bf3b3 P 90ef635255234a8d8998cfa7951f1e32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90ef635255234a8d8998cfa7951f1e32, State: Initialized, Role: FOLLOWER
I20260928 14:45:29.033741 23716 raft_consensus.cc:348] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45783 } } peers { permanent_uuid: "fa74e6c10d9241628165ee829514df21" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33867 } } peers { permanent_uuid: "90ef635255234a8d8998cfa7951f1e32" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38363 } }
I20260928 14:45:29.033792 23855 consensus_queue.cc:260] T 7c7d62a1e9d64474a3a898bd887bf3b3 P 90ef635255234a8d8998cfa7951f1e32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45783 } } peers { permanent_uuid: "fa74e6c10d9241628165ee829514df21" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33867 } } peers { permanent_uuid: "90ef635255234a8d8998cfa7951f1e32" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38363 } }
I20260928 14:45:29.033808 23716 raft_consensus.cc:374] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:45:29.033865 23716 raft_consensus.cc:729] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa74e6c10d9241628165ee829514df21, State: Initialized, Role: FOLLOWER
I20260928 14:45:29.033905 23855 ts_tablet_manager.cc:1428] T 7c7d62a1e9d64474a3a898bd887bf3b3 P 90ef635255234a8d8998cfa7951f1e32: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260928 14:45:29.033910 23716 consensus_queue.cc:260] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45783 } } peers { permanent_uuid: "fa74e6c10d9241628165ee829514df21" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33867 } } peers { permanent_uuid: "90ef635255234a8d8998cfa7951f1e32" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38363 } }
I20260928 14:45:29.033984 23965 heartbeater.cc:503] Master 127.9.33.126:46763 was elected leader, sending a full tablet report...
I20260928 14:45:29.033999 23716 ts_tablet_manager.cc:1428] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:29.036222 23831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:29.029706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:45783 (local address 127.0.0.1:35884)
0928 14:45:29.029820 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:29.029833 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:29.029913 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:29.032190 (+ 2277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:29.032191 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:29.032193 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:45:29.032254 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:29.032256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:29.033685 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:29.033686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:29.034858 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:29.034861 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:29.035927 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:29.035928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:29.035981 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:29.036002 (+ 21us) client_negotiation.cc:769] Sending connection context
0928 14:45:29.036116 (+ 114us) client_negotiation.cc:240] Negotiation successful
0928 14:45:29.036168 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260928 14:45:29.036226 24105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:29.030015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35884 (local address 127.9.33.67:45783)
0928 14:45:29.031954 (+ 1939us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:29.031959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:29.031972 (+ 13us) server_negotiation.cc:407] Connection header received
0928 14:45:29.032021 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:29.032024 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:29.032044 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:29.032159 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:45:29.032400 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:29.033561 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:29.034967 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:29.035857 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:29.035911 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:29.035956 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:29.036127 (+ 171us) server_negotiation.cc:299] Negotiation successful
0928 14:45:29.036160 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1892,"thread_start_us":120,"threads_started":1}
I20260928 14:45:29.037041 24033 tablet_service.cc:1467] Processing CreateTablet for tablet 7c7d62a1e9d64474a3a898bd887bf3b3 (DEFAULT_TABLE table=test-table [id=5aab3e0c3202443682c363221c017124]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:45:29.037338 24033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c7d62a1e9d64474a3a898bd887bf3b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:45:29.040504 24108 tablet_bootstrap.cc:492] T 7c7d62a1e9d64474a3a898bd887bf3b3 P b43bdfad8d3b4aa1bf499e9148871c0e: Bootstrap starting.
I20260928 14:45:29.041167 24108 tablet_bootstrap.cc:654] T 7c7d62a1e9d64474a3a898bd887bf3b3 P b43bdfad8d3b4aa1bf499e9148871c0e: Neither blocks nor log segments found. Creating new log.
I20260928 14:45:29.041625 24108 log.cc:824] T 7c7d62a1e9d64474a3a898bd887bf3b3 P b43bdfad8d3b4aa1bf499e9148871c0e: Log is configured to *not* fsync() on all Append() calls
I20260928 14:45:29.042347 24108 tablet_bootstrap.cc:492] T 7c7d62a1e9d64474a3a898bd887bf3b3 P b43bdfad8d3b4aa1bf499e9148871c0e: No bootstrap required, opened a new log
I20260928 14:45:29.042411 24108 ts_tablet_manager.cc:1397] T 7c7d62a1e9d64474a3a898bd887bf3b3 P b43bdfad8d3b4aa1bf499e9148871c0e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:45:29.044108 24108 raft_consensus.cc:348] T 7c7d62a1e9d64474a3a898bd887bf3b3 P b43bdfad8d3b4aa1bf499e9148871c0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45783 } } peers { permanent_uuid: "fa74e6c10d9241628165ee829514df21" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33867 } } peers { permanent_uuid: "90ef635255234a8d8998cfa7951f1e32" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38363 } }
I20260928 14:45:29.044304 24108 raft_consensus.cc:374] T 7c7d62a1e9d64474a3a898bd887bf3b3 P b43bdfad8d3b4aa1bf499e9148871c0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:45:29.044335 24108 raft_consensus.cc:729] T 7c7d62a1e9d64474a3a898bd887bf3b3 P b43bdfad8d3b4aa1bf499e9148871c0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b43bdfad8d3b4aa1bf499e9148871c0e, State: Initialized, Role: FOLLOWER
I20260928 14:45:29.044466 24108 consensus_queue.cc:260] T 7c7d62a1e9d64474a3a898bd887bf3b3 P b43bdfad8d3b4aa1bf499e9148871c0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45783 } } peers { permanent_uuid: "fa74e6c10d9241628165ee829514df21" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33867 } } peers { permanent_uuid: "90ef635255234a8d8998cfa7951f1e32" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38363 } }
I20260928 14:45:29.044744 24108 ts_tablet_manager.cc:1428] T 7c7d62a1e9d64474a3a898bd887bf3b3 P b43bdfad8d3b4aa1bf499e9148871c0e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260928 14:45:29.044781 24099 heartbeater.cc:503] Master 127.9.33.126:46763 was elected leader, sending a full tablet report...
I20260928 14:45:29.111101 24110 raft_consensus.cc:482] T 7c7d62a1e9d64474a3a898bd887bf3b3 P b43bdfad8d3b4aa1bf499e9148871c0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:45:29.111321 24110 raft_consensus.cc:504] T 7c7d62a1e9d64474a3a898bd887bf3b3 P b43bdfad8d3b4aa1bf499e9148871c0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45783 } } peers { permanent_uuid: "fa74e6c10d9241628165ee829514df21" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33867 } } peers { permanent_uuid: "90ef635255234a8d8998cfa7951f1e32" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38363 } }
I20260928 14:45:29.111811 24110 leader_election.cc:288] T 7c7d62a1e9d64474a3a898bd887bf3b3 P b43bdfad8d3b4aa1bf499e9148871c0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa74e6c10d9241628165ee829514df21 (127.9.33.65:33867), 90ef635255234a8d8998cfa7951f1e32 (127.9.33.66:38363)
I20260928 14:45:29.115151 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:29.112148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:57985 (local address 127.9.33.66:38363)
0928 14:45:29.112195 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:29.112197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:29.112326 (+ 129us) server_negotiation.cc:407] Connection header received
0928 14:45:29.112397 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:29.112397 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:29.112423 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:29.112488 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:45:29.112709 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:29.113220 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:29.114317 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:29.114767 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:29.114826 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:29.114940 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:29.115005 (+ 65us) server_negotiation.cc:299] Negotiation successful
0928 14:45:29.115054 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 14:45:29.115154 24101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:29.112035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:33867 (local address 127.9.33.67:34715)
0928 14:45:29.112119 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:29.112132 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:29.112193 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:29.112385 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:29.112386 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:29.112388 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:45:29.112467 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:29.112469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:29.113302 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:29.113304 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:29.114207 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:29.114210 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:29.114883 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:29.114884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:29.114966 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:29.114970 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:45:29.115027 (+ 57us) client_negotiation.cc:240] Negotiation successful
0928 14:45:29.115070 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63,"mutex_wait_us":90}
I20260928 14:45:29.115180 23832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:29.112088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:34715 (local address 127.9.33.65:33867)
0928 14:45:29.112154 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:29.112156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:29.112227 (+ 71us) server_negotiation.cc:407] Connection header received
0928 14:45:29.112263 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:29.112264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:29.112295 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:29.112371 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:45:29.112598 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:29.113211 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:29.114355 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:29.114771 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:29.114866 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:29.114945 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:29.115046 (+ 101us) server_negotiation.cc:299] Negotiation successful
0928 14:45:29.115096 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 14:45:29.115193 24111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:29.112048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:38363 (local address 127.9.33.67:57985)
0928 14:45:29.112244 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:29.112256 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:29.112322 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:29.112526 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:29.112527 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:29.112529 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:45:29.112620 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:29.112622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:29.113345 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:29.113351 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:29.114209 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:29.114211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:29.114864 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:29.114865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:29.114930 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:29.114935 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:45:29.114998 (+ 63us) client_negotiation.cc:240] Negotiation successful
0928 14:45:29.115072 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":27,"thread_start_us":108,"threads_started":1}
I20260928 14:45:29.115513 23918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c7d62a1e9d64474a3a898bd887bf3b3" candidate_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90ef635255234a8d8998cfa7951f1e32" is_pre_election: true
I20260928 14:45:29.115530 23778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c7d62a1e9d64474a3a898bd887bf3b3" candidate_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa74e6c10d9241628165ee829514df21" is_pre_election: true
I20260928 14:45:29.115696 23778 raft_consensus.cc:2454] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b43bdfad8d3b4aa1bf499e9148871c0e in term 0.
I20260928 14:45:29.115696 23918 raft_consensus.cc:2454] T 7c7d62a1e9d64474a3a898bd887bf3b3 P 90ef635255234a8d8998cfa7951f1e32 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b43bdfad8d3b4aa1bf499e9148871c0e in term 0.
I20260928 14:45:29.115968 23987 leader_election.cc:302] T 7c7d62a1e9d64474a3a898bd887bf3b3 P b43bdfad8d3b4aa1bf499e9148871c0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b43bdfad8d3b4aa1bf499e9148871c0e, fa74e6c10d9241628165ee829514df21; no voters:
I20260928 14:45:29.116164 24110 raft_consensus.cc:2781] T 7c7d62a1e9d64474a3a898bd887bf3b3 P b43bdfad8d3b4aa1bf499e9148871c0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:45:29.116230 24110 raft_consensus.cc:482] T 7c7d62a1e9d64474a3a898bd887bf3b3 P b43bdfad8d3b4aa1bf499e9148871c0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:45:29.116262 24110 raft_consensus.cc:3037] T 7c7d62a1e9d64474a3a898bd887bf3b3 P b43bdfad8d3b4aa1bf499e9148871c0e [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:45:29.117038 24110 raft_consensus.cc:504] T 7c7d62a1e9d64474a3a898bd887bf3b3 P b43bdfad8d3b4aa1bf499e9148871c0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45783 } } peers { permanent_uuid: "fa74e6c10d9241628165ee829514df21" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33867 } } peers { permanent_uuid: "90ef635255234a8d8998cfa7951f1e32" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38363 } }
I20260928 14:45:29.117264 24110 leader_election.cc:288] T 7c7d62a1e9d64474a3a898bd887bf3b3 P b43bdfad8d3b4aa1bf499e9148871c0e [CANDIDATE]: Term 1 election: Requested vote from peers fa74e6c10d9241628165ee829514df21 (127.9.33.65:33867), 90ef635255234a8d8998cfa7951f1e32 (127.9.33.66:38363)
I20260928 14:45:29.117363 23778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c7d62a1e9d64474a3a898bd887bf3b3" candidate_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa74e6c10d9241628165ee829514df21"
I20260928 14:45:29.117417 23918 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c7d62a1e9d64474a3a898bd887bf3b3" candidate_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90ef635255234a8d8998cfa7951f1e32"
I20260928 14:45:29.117466 23778 raft_consensus.cc:3037] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:45:29.117532 23918 raft_consensus.cc:3037] T 7c7d62a1e9d64474a3a898bd887bf3b3 P 90ef635255234a8d8998cfa7951f1e32 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:45:29.118258 23778 raft_consensus.cc:2454] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b43bdfad8d3b4aa1bf499e9148871c0e in term 1.
I20260928 14:45:29.118263 23918 raft_consensus.cc:2454] T 7c7d62a1e9d64474a3a898bd887bf3b3 P 90ef635255234a8d8998cfa7951f1e32 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b43bdfad8d3b4aa1bf499e9148871c0e in term 1.
I20260928 14:45:29.118449 23986 leader_election.cc:302] T 7c7d62a1e9d64474a3a898bd887bf3b3 P b43bdfad8d3b4aa1bf499e9148871c0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90ef635255234a8d8998cfa7951f1e32, b43bdfad8d3b4aa1bf499e9148871c0e; no voters:
I20260928 14:45:29.118597 24110 raft_consensus.cc:2781] T 7c7d62a1e9d64474a3a898bd887bf3b3 P b43bdfad8d3b4aa1bf499e9148871c0e [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:45:29.118749 24110 raft_consensus.cc:686] T 7c7d62a1e9d64474a3a898bd887bf3b3 P b43bdfad8d3b4aa1bf499e9148871c0e [term 1 LEADER]: Becoming Leader. State: Replica: b43bdfad8d3b4aa1bf499e9148871c0e, State: Running, Role: LEADER
I20260928 14:45:29.118853 24110 consensus_queue.cc:237] T 7c7d62a1e9d64474a3a898bd887bf3b3 P b43bdfad8d3b4aa1bf499e9148871c0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45783 } } peers { permanent_uuid: "fa74e6c10d9241628165ee829514df21" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33867 } } peers { permanent_uuid: "90ef635255234a8d8998cfa7951f1e32" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38363 } }
I20260928 14:45:29.119799 23635 catalog_manager.cc:5495] T 7c7d62a1e9d64474a3a898bd887bf3b3 P b43bdfad8d3b4aa1bf499e9148871c0e reported cstate change: term changed from 0 to 1, leader changed from <none> to b43bdfad8d3b4aa1bf499e9148871c0e (127.9.33.67). New cstate: current_term: 1 leader_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa74e6c10d9241628165ee829514df21" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90ef635255234a8d8998cfa7951f1e32" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38363 } health_report { overall_health: UNKNOWN } } }
W20260928 14:45:29.122804 23966 tablet.cc:2369] T 7c7d62a1e9d64474a3a898bd887bf3b3 P 90ef635255234a8d8998cfa7951f1e32: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:45:29.129143 24105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:29.126482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35900 (local address 127.9.33.67:45783)
0928 14:45:29.126538 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:29.126540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:29.126547 (+ 7us) server_negotiation.cc:407] Connection header received
0928 14:45:29.126586 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:29.126586 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:29.126606 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:29.126674 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:45:29.126787 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:29.127269 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:29.128244 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:29.128475 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:29.128548 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:29.128741 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:45:29.128935 (+ 194us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:45:29.129014 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:29.129034 (+ 20us) server_negotiation.cc:299] Negotiation successful
0928 14:45:29.129055 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 14:45:29.131273 23778 raft_consensus.cc:1261] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [term 1 FOLLOWER]: Refusing update from remote peer b43bdfad8d3b4aa1bf499e9148871c0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:45:29.131297 23918 raft_consensus.cc:1261] T 7c7d62a1e9d64474a3a898bd887bf3b3 P 90ef635255234a8d8998cfa7951f1e32 [term 1 FOLLOWER]: Refusing update from remote peer b43bdfad8d3b4aa1bf499e9148871c0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:45:29.131708 24110 consensus_queue.cc:1035] T 7c7d62a1e9d64474a3a898bd887bf3b3 P b43bdfad8d3b4aa1bf499e9148871c0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "90ef635255234a8d8998cfa7951f1e32" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:45:29.131981 24112 consensus_queue.cc:1035] T 7c7d62a1e9d64474a3a898bd887bf3b3 P b43bdfad8d3b4aa1bf499e9148871c0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa74e6c10d9241628165ee829514df21" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:45:29.134371 24115 mvcc.cc:205] Tried to move back new op lower bound from 7334325162518122496 to 7334325162471641088. Current Snapshot: MvccSnapshot[applied={T|T < 7334325162518122496}]
I20260928 14:45:29.134662 24114 mvcc.cc:205] Tried to move back new op lower bound from 7334325162518122496 to 7334325162471641088. Current Snapshot: MvccSnapshot[applied={T|T < 7334325162518122496 or (T in {7334325162518122496})}]
I20260928 14:45:29.137315 23635 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:41104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:45:29.137456 23635 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:41104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:45:29.138322 23635 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043: Sending DeleteTablet for 3 replicas of tablet 7c7d62a1e9d64474a3a898bd887bf3b3
I20260928 14:45:29.138655 23758 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c7d62a1e9d64474a3a898bd887bf3b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:45:29 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:49340
I20260928 14:45:29.138784 23898 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c7d62a1e9d64474a3a898bd887bf3b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:45:29 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:33008
I20260928 14:45:29.138886 23833 tablet_replica.cc:330] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21: stopping tablet replica
I20260928 14:45:29.138864 24033 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c7d62a1e9d64474a3a898bd887bf3b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:45:29 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:35884
I20260928 14:45:29.138953 23833 raft_consensus.cc:2229] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:45:29.139024 23833 raft_consensus.cc:2258] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:45:29.139110 23970 tablet_replica.cc:330] T 7c7d62a1e9d64474a3a898bd887bf3b3 P 90ef635255234a8d8998cfa7951f1e32: stopping tablet replica
I20260928 14:45:29.139132 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 23674
I20260928 14:45:29.139173 23970 raft_consensus.cc:2229] T 7c7d62a1e9d64474a3a898bd887bf3b3 P 90ef635255234a8d8998cfa7951f1e32 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:45:29.139230 23970 raft_consensus.cc:2258] T 7c7d62a1e9d64474a3a898bd887bf3b3 P 90ef635255234a8d8998cfa7951f1e32 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:45:29.139142 24124 tablet_replica.cc:330] T 7c7d62a1e9d64474a3a898bd887bf3b3 P b43bdfad8d3b4aa1bf499e9148871c0e: stopping tablet replica
I20260928 14:45:29.139391 24124 raft_consensus.cc:2229] T 7c7d62a1e9d64474a3a898bd887bf3b3 P b43bdfad8d3b4aa1bf499e9148871c0e [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:45:29.139539 24124 raft_consensus.cc:2258] T 7c7d62a1e9d64474a3a898bd887bf3b3 P b43bdfad8d3b4aa1bf499e9148871c0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:45:29.140085 23970 ts_tablet_manager.cc:1905] T 7c7d62a1e9d64474a3a898bd887bf3b3 P 90ef635255234a8d8998cfa7951f1e32: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:45:29.140210 24124 ts_tablet_manager.cc:1905] T 7c7d62a1e9d64474a3a898bd887bf3b3 P b43bdfad8d3b4aa1bf499e9148871c0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:45:29.141732 23970 ts_tablet_manager.cc:1918] T 7c7d62a1e9d64474a3a898bd887bf3b3 P 90ef635255234a8d8998cfa7951f1e32: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:45:29.141803 23970 log.cc:1196] T 7c7d62a1e9d64474a3a898bd887bf3b3 P 90ef635255234a8d8998cfa7951f1e32: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/wal/wals/7c7d62a1e9d64474a3a898bd887bf3b3
I20260928 14:45:29.142038 23970 ts_tablet_manager.cc:1939] T 7c7d62a1e9d64474a3a898bd887bf3b3 P 90ef635255234a8d8998cfa7951f1e32: Deleting consensus metadata
I20260928 14:45:29.142495 24124 ts_tablet_manager.cc:1918] T 7c7d62a1e9d64474a3a898bd887bf3b3 P b43bdfad8d3b4aa1bf499e9148871c0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:45:29.142638 24124 log.cc:1196] T 7c7d62a1e9d64474a3a898bd887bf3b3 P b43bdfad8d3b4aa1bf499e9148871c0e: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/wal/wals/7c7d62a1e9d64474a3a898bd887bf3b3
I20260928 14:45:29.142964 24124 ts_tablet_manager.cc:1939] T 7c7d62a1e9d64474a3a898bd887bf3b3 P b43bdfad8d3b4aa1bf499e9148871c0e: Deleting consensus metadata
I20260928 14:45:29.143487 23622 catalog_manager.cc:4841] TS b43bdfad8d3b4aa1bf499e9148871c0e (127.9.33.67:45783): tablet 7c7d62a1e9d64474a3a898bd887bf3b3 (table test-table [id=5aab3e0c3202443682c363221c017124]) successfully deleted
I20260928 14:45:29.143925 23620 catalog_manager.cc:4841] TS 90ef635255234a8d8998cfa7951f1e32 (127.9.33.66:38363): tablet 7c7d62a1e9d64474a3a898bd887bf3b3 (table test-table [id=5aab3e0c3202443682c363221c017124]) successfully deleted
W20260928 14:45:29.146464 23621 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.33.65:33867 (error 108)
I20260928 14:45:29.147004 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 23830
I20260928 14:45:29.147037 23831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:29.146894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:33867 (local address 127.0.0.1:49348)
0928 14:45:29.146956 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:29.147001 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.65:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260928 14:45:29.148227 23621 catalog_manager.cc:4584] TS fa74e6c10d9241628165ee829514df21 (127.9.33.65:33867): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7c7d62a1e9d64474a3a898bd887bf3b3: Network error: Client connection negotiation failed: client connection to 127.9.33.65:33867: connect: Connection refused (error 111)
I20260928 14:45:29.153678 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 23968
I20260928 14:45:29.158694 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 23603
I20260928 14:45:29.165031 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:46763
--webserver_interface=127.9.33.126
--webserver_port=46303
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:35485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:46763
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:45:29.249840 24126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:45:29.250097 24126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:45:29.250138 24126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:45:29.251399 24126 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:45:29.251477 24126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:45:29.251502 24126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:45:29.251533 24126 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:45:29.251564 24126 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:45:29.253204 24126 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:35485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:46763
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:46763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=46303
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24126
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:45:29.253525 24126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:45:29.253772 24126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:45:29.258945 24132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:29.258945 24134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:29.259025 24131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:45:29.259366 24126 server_base.cc:1034] running on GCE node
I20260928 14:45:29.259625 24126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:45:29.259982 24126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:45:29.261173 24126 hybrid_clock.cc:648] HybridClock initialized: now 1790606729261145 us; error 36 us; skew 500 ppm
Sep 28 14:45:29 dist-test-slave-pv0w krb5kdc[23056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606729, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:45:29.263975 24126 init.cc:377] Logged in from keytab as kudu/127.9.33.126@KRBTEST.COM (short username kudu)
I20260928 14:45:29.264524 24126 webserver.cc:460] Webserver started at http://127.9.33.126:46303/ using document root <none> and password file <none>
I20260928 14:45:29.264796 24126 fs_manager.cc:359] Metadata directory not provided
I20260928 14:45:29.264863 24126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:45:29.266692 24126 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:45:29.267536 24141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:29.267774 24126 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:45:29.267871 24126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "ad4350e8668941ea9fa505f99e327043"
format_stamp: "Formatted at 2026-09-28 14:45:27 on dist-test-slave-pv0w"
server_key: "1e3c3ed5b194beb1c194636d9803ebf3"
server_key_iv: "3fc69158bdc24fcfd5336afb59720978"
server_key_version: "encryptionkey@0"
I20260928 14:45:29.268231 24126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:45:29.277446 24126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:45:29.279593 24126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:45:29.279779 24126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:45:29.283627 24126 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:46763
I20260928 14:45:29.283675 24193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:46763 every 8 connection(s)
I20260928 14:45:29.284113 24126 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:45:29.285727 24194 sys_catalog.cc:263] Verifying existing consensus state
I20260928 14:45:29.286437 24194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043: Bootstrap starting.
I20260928 14:45:29.290346 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 24126
I20260928 14:45:29.290632 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.65:33867
--local_ip_for_outbound_sockets=127.9.33.65
--tserver_master_addrs=127.9.33.126:46763
--webserver_port=46089
--webserver_interface=127.9.33.65
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:35485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:45:29.295552 24194 log.cc:824] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043: Log is configured to *not* fsync() on all Append() calls
I20260928 14:45:29.297951 24194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 14:45:29.298295 24194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043: Bootstrap complete.
I20260928 14:45:29.300928 24194 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad4350e8668941ea9fa505f99e327043" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 46763 } }
I20260928 14:45:29.301208 24194 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad4350e8668941ea9fa505f99e327043, State: Initialized, Role: FOLLOWER
I20260928 14:45:29.301436 24194 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad4350e8668941ea9fa505f99e327043" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 46763 } }
I20260928 14:45:29.301540 24194 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:45:29.301595 24194 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:45:29.301645 24194 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [term 2 FOLLOWER]: Advancing to term 3
I20260928 14:45:29.302706 24194 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad4350e8668941ea9fa505f99e327043" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 46763 } }
I20260928 14:45:29.302832 24194 leader_election.cc:302] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad4350e8668941ea9fa505f99e327043; no voters:
I20260928 14:45:29.303087 24194 leader_election.cc:288] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260928 14:45:29.303126 24198 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [term 3 FOLLOWER]: Leader election won for term 3
I20260928 14:45:29.303252 24198 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [term 3 LEADER]: Becoming Leader. State: Replica: ad4350e8668941ea9fa505f99e327043, State: Running, Role: LEADER
I20260928 14:45:29.303354 24198 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad4350e8668941ea9fa505f99e327043" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 46763 } }
I20260928 14:45:29.303400 24194 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:45:29.303634 24198 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad4350e8668941ea9fa505f99e327043. Latest consensus state: current_term: 3 leader_uuid: "ad4350e8668941ea9fa505f99e327043" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad4350e8668941ea9fa505f99e327043" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 46763 } } }
I20260928 14:45:29.303709 24198 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [sys.catalog]: This master's current role is: LEADER
I20260928 14:45:29.303640 24199 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "ad4350e8668941ea9fa505f99e327043" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad4350e8668941ea9fa505f99e327043" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 46763 } } }
I20260928 14:45:29.303787 24199 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad4350e8668941ea9fa505f99e327043 [sys.catalog]: This master's current role is: LEADER
I20260928 14:45:29.303968 24202 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:45:29.305080 24202 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:45:29.305205 24202 catalog_manager.cc:1259] Loaded cluster ID: d38f0742c7664947ac2e98492bc58dbd
I20260928 14:45:29.305248 24202 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260928 14:45:29.305686 24202 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f0f4da1e9ad google::LogMessage::Fail() at ??:0
@ 0x7f0f4da2188a google::LogMessage::SendToLog() at ??:0
@ 0x7f0f4da1e4de google::LogMessage::Flush() at ??:0
@ 0x7f0f4da1fa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f0f5593f0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f0f559488b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f0f4f8ec0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f0f4f8e1954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f0f507456db start_thread at ??:0
@ 0x7f0f4c51c61f clone at ??:0
W20260928 14:45:29.410456 24196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:45:29.410697 24196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:45:29.410732 24196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:45:29.412127 24196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:45:29.412180 24196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:45:29.412212 24196 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.65
I20260928 14:45:29.413827 24196 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:35485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.65:33867
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.33.65
--webserver_port=46089
--tserver_master_addrs=127.9.33.126:46763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.65
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:45:29.414094 24196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:45:29.414356 24196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:45:29.420729 24230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:29.420709 24231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:29.421007 24233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:45:29.420975 24196 server_base.cc:1034] running on GCE node
I20260928 14:45:29.421274 24196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:45:29.421579 24196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:45:29.422747 24196 hybrid_clock.cc:648] HybridClock initialized: now 1790606729422716 us; error 48 us; skew 500 ppm
Sep 28 14:45:29 dist-test-slave-pv0w krb5kdc[23056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606729, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:45:29.425660 24196 init.cc:377] Logged in from keytab as kudu/127.9.33.65@KRBTEST.COM (short username kudu)
I20260928 14:45:29.426239 24196 webserver.cc:460] Webserver started at http://127.9.33.65:46089/ using document root <none> and password file <none>
I20260928 14:45:29.426535 24196 fs_manager.cc:359] Metadata directory not provided
I20260928 14:45:29.426612 24196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:45:29.428093 24196 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260928 14:45:29.428894 24240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:29.429152 24196 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:45:29.429260 24196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/wal
uuid: "fa74e6c10d9241628165ee829514df21"
format_stamp: "Formatted at 2026-09-28 14:45:27 on dist-test-slave-pv0w"
server_key: "6d2358cd2f05abf78e5a16810320fbb7"
server_key_iv: "f5cb5c9ee5ac786dda0ec001022df31f"
server_key_version: "encryptionkey@0"
I20260928 14:45:29.429737 24196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:45:29.450495 24196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:45:29.453439 24196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:45:29.453604 24196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:45:29.453892 24196 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:45:29.454471 24247 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260928 14:45:29.456159 24196 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260928 14:45:29.456221 24196 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260928 14:45:29.456257 24196 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260928 14:45:29.457170 24196 ts_tablet_manager.cc:610] Registered 1 tablets
I20260928 14:45:29.457217 24196 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260928 14:45:29.457350 24247 tablet_bootstrap.cc:492] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21: Bootstrap starting.
I20260928 14:45:29.464491 24196 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.65:33867
I20260928 14:45:29.464556 24354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.65:33867 every 8 connection(s)
I20260928 14:45:29.464977 24196 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
I20260928 14:45:29.467180 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 24196
I20260928 14:45:29.467474 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.66:38363
--local_ip_for_outbound_sockets=127.9.33.66
--tserver_master_addrs=127.9.33.126:46763
--webserver_port=44581
--webserver_interface=127.9.33.66
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:35485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:45:29.468519 24247 log.cc:824] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21: Log is configured to *not* fsync() on all Append() calls
I20260928 14:45:29.471395 24247 tablet_bootstrap.cc:492] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 14:45:29.471757 24247 tablet_bootstrap.cc:492] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21: Bootstrap complete.
I20260928 14:45:29.473013 24247 ts_tablet_manager.cc:1397] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.009s
I20260928 14:45:29.474485 24247 raft_consensus.cc:348] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45783 } } peers { permanent_uuid: "fa74e6c10d9241628165ee829514df21" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33867 } } peers { permanent_uuid: "90ef635255234a8d8998cfa7951f1e32" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38363 } }
I20260928 14:45:29.474828 24247 raft_consensus.cc:729] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa74e6c10d9241628165ee829514df21, State: Initialized, Role: FOLLOWER
I20260928 14:45:29.474982 24247 consensus_queue.cc:260] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45783 } } peers { permanent_uuid: "fa74e6c10d9241628165ee829514df21" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33867 } } peers { permanent_uuid: "90ef635255234a8d8998cfa7951f1e32" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38363 } }
I20260928 14:45:29.475291 24247 ts_tablet_manager.cc:1428] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
W20260928 14:45:29.525496 24357 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:45:29.465706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.65:37341)
0928 14:45:29.465910 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:29.465968 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:29.466624 (+ 656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:29.525256 (+ 58632us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":87,"threads_started":1}
W20260928 14:45:29.525754 24355 heartbeater.cc:644] Failed to heartbeat to 127.9.33.126:46763 (0 consecutive failures): Network error: Failed to ping master at 127.9.33.126:46763: Client connection negotiation failed: client connection to 127.9.33.126:46763: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260928 14:45:29.526145 24357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:29.526033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.65:52637)
0928 14:45:29.526088 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:29.526126 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260928 14:45:29.526463 24357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:29.526370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.65:54951)
0928 14:45:29.526414 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:29.526437 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260928 14:45:29.526665 24355 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 14:45:29.553368 24359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:45:29.553610 24359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:45:29.553654 24359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:45:29.554889 24359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:45:29.554960 24359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:45:29.555009 24359 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.66
I20260928 14:45:29.556639 24359 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:35485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.66:38363
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.33.66
--webserver_port=44581
--tserver_master_addrs=127.9.33.126:46763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24359
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.66
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:45:29.556918 24359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:45:29.557163 24359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:45:29.562479 24366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:29.562500 24365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:29.562496 24368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:45:29.562795 24359 server_base.cc:1034] running on GCE node
I20260928 14:45:29.563030 24359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:45:29.563302 24359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:45:29.564446 24359 hybrid_clock.cc:648] HybridClock initialized: now 1790606729564433 us; error 29 us; skew 500 ppm
Sep 28 14:45:29 dist-test-slave-pv0w krb5kdc[23056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606729, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:45:29.567097 24359 init.cc:377] Logged in from keytab as kudu/127.9.33.66@KRBTEST.COM (short username kudu)
I20260928 14:45:29.567648 24359 webserver.cc:460] Webserver started at http://127.9.33.66:44581/ using document root <none> and password file <none>
I20260928 14:45:29.567932 24359 fs_manager.cc:359] Metadata directory not provided
I20260928 14:45:29.567981 24359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:45:29.569423 24359 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:45:29.570040 24375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:29.570221 24359 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:45:29.570302 24359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/wal
uuid: "90ef635255234a8d8998cfa7951f1e32"
format_stamp: "Formatted at 2026-09-28 14:45:28 on dist-test-slave-pv0w"
server_key: "8ccbe6dc6913e0350f1ccc4933697848"
server_key_iv: "5ed48d52c22b6c84de3afc350ee33f4c"
server_key_version: "encryptionkey@0"
I20260928 14:45:29.570684 24359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:45:29.581852 24359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:45:29.584348 24359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:45:29.584517 24359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:45:29.584739 24359 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:45:29.585049 24359 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:45:29.585083 24359 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:29.585125 24359 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:45:29.585152 24359 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:29.590714 24359 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.66:38363
I20260928 14:45:29.590796 24488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.66:38363 every 8 connection(s)
I20260928 14:45:29.591158 24359 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
I20260928 14:45:29.592278 24491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:29.591915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.66:60807)
0928 14:45:29.592075 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:29.592143 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":80,"threads_started":1}
W20260928 14:45:29.592705 24489 heartbeater.cc:644] Failed to heartbeat to 127.9.33.126:46763 (0 consecutive failures): Network error: Failed to ping master at 127.9.33.126:46763: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
I20260928 14:45:29.592991 24491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:29.592900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.66:55999)
0928 14:45:29.592947 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:29.592974 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260928 14:45:29.593245 24491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:29.593174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.66:39789)
0928 14:45:29.593212 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:29.593231 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260928 14:45:29.593367 24489 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260928 14:45:29.593421 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 24359
I20260928 14:45:29.593686 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.67:45783
--local_ip_for_outbound_sockets=127.9.33.67
--tserver_master_addrs=127.9.33.126:46763
--webserver_port=46637
--webserver_interface=127.9.33.67
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:35485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:45:29.675980 24492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:45:29.676219 24492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:45:29.676260 24492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:45:29.677402 24492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:45:29.677466 24492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:45:29.677512 24492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.67
I20260928 14:45:29.678969 24492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:35485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.67:45783
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.33.67
--webserver_port=46637
--tserver_master_addrs=127.9.33.126:46763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.67
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:45:29.679240 24492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:45:29.679476 24492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:45:29.684700 24497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:29.684723 24498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:29.684710 24500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:45:29.684926 24492 server_base.cc:1034] running on GCE node
I20260928 14:45:29.685135 24492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:45:29.685379 24492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:45:29.686539 24492 hybrid_clock.cc:648] HybridClock initialized: now 1790606729686536 us; error 38 us; skew 500 ppm
Sep 28 14:45:29 dist-test-slave-pv0w krb5kdc[23056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606729, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:45:29.689039 24492 init.cc:377] Logged in from keytab as kudu/127.9.33.67@KRBTEST.COM (short username kudu)
I20260928 14:45:29.689514 24492 webserver.cc:460] Webserver started at http://127.9.33.67:46637/ using document root <none> and password file <none>
I20260928 14:45:29.689741 24492 fs_manager.cc:359] Metadata directory not provided
I20260928 14:45:29.689810 24492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:45:29.691176 24492 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:45:29.691864 24507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:29.692040 24492 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260928 14:45:29.692137 24492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/wal
uuid: "b43bdfad8d3b4aa1bf499e9148871c0e"
format_stamp: "Formatted at 2026-09-28 14:45:28 on dist-test-slave-pv0w"
server_key: "01e19ff0a4f3acc9e3040683e1657e1d"
server_key_iv: "e35e4b450a85b2eae000a3c8b2da56df"
server_key_version: "encryptionkey@0"
I20260928 14:45:29.692507 24492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:45:29.700536 24492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:45:29.702435 24492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:45:29.702598 24492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:45:29.702854 24492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:45:29.703158 24492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:45:29.703212 24492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:29.703258 24492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:45:29.703284 24492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:29.709398 24492 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.67:45783
I20260928 14:45:29.709448 24620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.67:45783 every 8 connection(s)
I20260928 14:45:29.709794 24492 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
I20260928 14:45:29.710856 24623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:29.710436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.67:35419)
0928 14:45:29.710594 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:29.710712 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":73,"threads_started":1}
W20260928 14:45:29.711290 24621 heartbeater.cc:644] Failed to heartbeat to 127.9.33.126:46763 (0 consecutive failures): Network error: Failed to ping master at 127.9.33.126:46763: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
I20260928 14:45:29.711601 24623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:29.711503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.67:45975)
0928 14:45:29.711553 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:29.711580 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260928 14:45:29.711923 24623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:29.711848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.67:37389)
0928 14:45:29.711886 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:29.711908 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260928 14:45:29.712064 24621 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260928 14:45:29.718730 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 24492
I20260928 14:45:30.527423 24625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:30.527070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.65:54437)
0928 14:45:30.527305 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:30.527354 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":97,"threads_started":1}
I20260928 14:45:30.593895 24626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:30.593684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.66:37487)
0928 14:45:30.593810 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:30.593845 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":66,"threads_started":1}
I20260928 14:45:30.712709 24627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:30.712460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.67:42717)
0928 14:45:30.712597 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:30.712641 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":61,"threads_started":1}
I20260928 14:45:30.715552 24628 raft_consensus.cc:482] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:45:30.715762 24628 raft_consensus.cc:504] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45783 } } peers { permanent_uuid: "fa74e6c10d9241628165ee829514df21" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33867 } } peers { permanent_uuid: "90ef635255234a8d8998cfa7951f1e32" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38363 } }
I20260928 14:45:30.716126 24628 leader_election.cc:288] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b43bdfad8d3b4aa1bf499e9148871c0e (127.9.33.67:45783), 90ef635255234a8d8998cfa7951f1e32 (127.9.33.66:38363)
Sep 28 14:45:30 dist-test-slave-pv0w krb5kdc[23056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606729, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for kudu/127.9.33.67@KRBTEST.COM
Sep 28 14:45:30 dist-test-slave-pv0w krb5kdc[23056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606729, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for kudu/127.9.33.66@KRBTEST.COM
I20260928 14:45:30.732010 24629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:30.716495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:42875 (local address 127.9.33.66:38363)
0928 14:45:30.716683 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:30.716700 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:30.716752 (+ 52us) server_negotiation.cc:407] Connection header received
0928 14:45:30.717245 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:30.717248 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:30.717297 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:30.717451 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:30.718348 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:30.719221 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:30.720326 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:30.720745 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:30.720881 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:30.728597 (+ 7716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:30.728602 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:30.728602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:30.728617 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:30.730353 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:30.730737 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:30.730741 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:30.730741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:30.730845 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:30.731057 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:30.731058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:30.731058 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:30.731327 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:30.731416 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:30.731742 (+ 326us) server_negotiation.cc:299] Negotiation successful
0928 14:45:30.731887 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":58,"threads_started":1}
I20260928 14:45:30.731997 24625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:30.716335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:38363 (local address 127.9.33.65:42875)
0928 14:45:30.716405 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:30.716418 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:30.716495 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:30.717488 (+ 993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:30.717491 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:30.717916 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:45:30.718169 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:30.718177 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:30.719382 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:30.719383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:30.720230 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:30.720233 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:30.720858 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:30.720859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:30.720975 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:30.726227 (+ 5252us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:45:30.726239 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:45:30.728443 (+ 2204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:45:30.730467 (+ 2024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:30.730469 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:30.730469 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:30.730649 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:30.730924 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:30.730925 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:30.730925 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:30.730988 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:30.731410 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:45:30.731417 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:45:30.731550 (+ 133us) client_negotiation.cc:769] Sending connection context
0928 14:45:30.731653 (+ 103us) client_negotiation.cc:240] Negotiation successful
0928 14:45:30.731900 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":74}
I20260928 14:45:30.732026 24630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:30.716515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:38181 (local address 127.9.33.67:45783)
0928 14:45:30.716681 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:30.716698 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:30.716770 (+ 72us) server_negotiation.cc:407] Connection header received
0928 14:45:30.717231 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:30.717234 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:30.717282 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:30.717430 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:30.718343 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:30.719222 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:30.720380 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:30.720792 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:30.720923 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:30.728248 (+ 7325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:30.728253 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:30.728253 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:30.728265 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:30.730173 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:30.730676 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:30.730680 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:30.730680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:30.730784 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:30.731050 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:30.731051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:30.731051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:30.731324 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:30.731416 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:30.731750 (+ 334us) server_negotiation.cc:299] Negotiation successful
0928 14:45:30.731890 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":103,"threads_started":1}
I20260928 14:45:30.732120 24631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:30.716343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:45783 (local address 127.9.33.65:38181)
0928 14:45:30.716705 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:30.716716 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:30.716786 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:30.717489 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:30.717492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:30.717916 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:45:30.718170 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:30.718177 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:30.719339 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:30.719340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:30.720262 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:30.720265 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:30.720893 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:30.720895 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:30.720975 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:30.726222 (+ 5247us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:45:30.726234 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:45:30.728091 (+ 1857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:45:30.730286 (+ 2195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:30.730295 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:30.730298 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:30.730568 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:30.730858 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:30.730859 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:30.730860 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:30.730971 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:30.731406 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:45:30.731414 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:45:30.731550 (+ 136us) client_negotiation.cc:769] Sending connection context
0928 14:45:30.731655 (+ 105us) client_negotiation.cc:240] Negotiation successful
0928 14:45:30.731984 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"mutex_wait_us":432,"thread_start_us":57,"threads_started":1}
W20260928 14:45:30.733819 24241 leader_election.cc:341] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 90ef635255234a8d8998cfa7951f1e32 (127.9.33.66:38363): Not found: Tablet not found: 7c7d62a1e9d64474a3a898bd887bf3b3
W20260928 14:45:30.733901 24243 leader_election.cc:341] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b43bdfad8d3b4aa1bf499e9148871c0e (127.9.33.67:45783): Not found: Tablet not found: 7c7d62a1e9d64474a3a898bd887bf3b3
I20260928 14:45:30.733968 24243 leader_election.cc:302] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa74e6c10d9241628165ee829514df21; no voters: 90ef635255234a8d8998cfa7951f1e32, b43bdfad8d3b4aa1bf499e9148871c0e
I20260928 14:45:30.734066 24628 raft_consensus.cc:2726] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 14:45:31.528218 24632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:31.527991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.65:38099)
0928 14:45:31.528124 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:31.528168 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":82,"threads_started":1}
I20260928 14:45:31.594696 24633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:31.594438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.66:51937)
0928 14:45:31.594597 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:31.594647 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":65,"threads_started":1}
I20260928 14:45:31.713542 24634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:31.713315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.67:50919)
0928 14:45:31.713452 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:31.713497 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":63,"threads_started":1}
I20260928 14:45:32.529037 24635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:32.528814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.65:59037)
0928 14:45:32.528947 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:32.528987 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":71,"threads_started":1}
I20260928 14:45:32.595500 24636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:32.595280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.66:35535)
0928 14:45:32.595419 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:32.595459 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":62,"threads_started":1}
I20260928 14:45:32.702351 24637 raft_consensus.cc:482] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:45:32.702471 24637 raft_consensus.cc:504] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45783 } } peers { permanent_uuid: "fa74e6c10d9241628165ee829514df21" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33867 } } peers { permanent_uuid: "90ef635255234a8d8998cfa7951f1e32" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38363 } }
I20260928 14:45:32.702692 24637 leader_election.cc:288] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b43bdfad8d3b4aa1bf499e9148871c0e (127.9.33.67:45783), 90ef635255234a8d8998cfa7951f1e32 (127.9.33.66:38363)
W20260928 14:45:32.703109 24241 leader_election.cc:341] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 90ef635255234a8d8998cfa7951f1e32 (127.9.33.66:38363): Not found: Tablet not found: 7c7d62a1e9d64474a3a898bd887bf3b3
W20260928 14:45:32.703400 24243 leader_election.cc:341] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b43bdfad8d3b4aa1bf499e9148871c0e (127.9.33.67:45783): Not found: Tablet not found: 7c7d62a1e9d64474a3a898bd887bf3b3
I20260928 14:45:32.703469 24243 leader_election.cc:302] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa74e6c10d9241628165ee829514df21; no voters: 90ef635255234a8d8998cfa7951f1e32, b43bdfad8d3b4aa1bf499e9148871c0e
I20260928 14:45:32.703541 24637 raft_consensus.cc:2726] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 14:45:32.714375 24638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:32.714134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.67:55333)
0928 14:45:32.714271 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:32.714309 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":72,"threads_started":1}
I20260928 14:45:33.529863 24639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:33.529589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.65:36917)
0928 14:45:33.529765 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:33.529813 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":71,"threads_started":1}
I20260928 14:45:33.596261 24640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:33.596025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.66:39013)
0928 14:45:33.596169 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:33.596215 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":65,"threads_started":1}
I20260928 14:45:33.715130 24641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:33.714914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.67:39679)
0928 14:45:33.715048 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:33.715084 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":73,"threads_started":1}
I20260928 14:45:34.530694 24642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:34.530425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.65:44393)
0928 14:45:34.530590 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:34.530631 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":75,"threads_started":1}
I20260928 14:45:34.541333 24643 raft_consensus.cc:482] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:45:34.541437 24643 raft_consensus.cc:504] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45783 } } peers { permanent_uuid: "fa74e6c10d9241628165ee829514df21" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33867 } } peers { permanent_uuid: "90ef635255234a8d8998cfa7951f1e32" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38363 } }
I20260928 14:45:34.541605 24643 leader_election.cc:288] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b43bdfad8d3b4aa1bf499e9148871c0e (127.9.33.67:45783), 90ef635255234a8d8998cfa7951f1e32 (127.9.33.66:38363)
W20260928 14:45:34.541991 24243 leader_election.cc:341] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b43bdfad8d3b4aa1bf499e9148871c0e (127.9.33.67:45783): Not found: Tablet not found: 7c7d62a1e9d64474a3a898bd887bf3b3
W20260928 14:45:34.542068 24241 leader_election.cc:341] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 90ef635255234a8d8998cfa7951f1e32 (127.9.33.66:38363): Not found: Tablet not found: 7c7d62a1e9d64474a3a898bd887bf3b3
I20260928 14:45:34.542098 24243 leader_election.cc:302] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa74e6c10d9241628165ee829514df21; no voters: 90ef635255234a8d8998cfa7951f1e32, b43bdfad8d3b4aa1bf499e9148871c0e
I20260928 14:45:34.542186 24643 raft_consensus.cc:2726] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 14:45:34.597011 24644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:34.596778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.66:53853)
0928 14:45:34.596921 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:34.596963 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":77,"threads_started":1}
I20260928 14:45:34.715951 24645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:34.715654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.67:35823)
0928 14:45:34.715855 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:34.715902 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":142,"threads_started":1}
I20260928 14:45:35.531587 24646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:35.531298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.65:35447)
0928 14:45:35.531496 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:35.531539 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":117,"threads_started":1}
I20260928 14:45:35.597944 24647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:35.597648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.66:48975)
0928 14:45:35.597829 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:35.597879 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":106,"threads_started":1}
I20260928 14:45:35.716934 24648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:35.716612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.67:39263)
0928 14:45:35.716809 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:35.716866 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":88,"threads_started":1}
I20260928 14:45:36.532503 24649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:36.532204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.65:33723)
0928 14:45:36.532382 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:36.532434 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":92,"threads_started":1}
I20260928 14:45:36.598944 24650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:36.598632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.66:57141)
0928 14:45:36.598819 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:36.598874 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":86,"threads_started":1}
I20260928 14:45:36.717756 24651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:36.717535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.67:40383)
0928 14:45:36.717671 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:36.717702 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":69,"threads_started":1}
I20260928 14:45:37.533449 24652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:37.533174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.65:55825)
0928 14:45:37.533348 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:37.533398 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":95,"threads_started":1}
I20260928 14:45:37.599929 24653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:37.599571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.66:54261)
0928 14:45:37.599820 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:37.599868 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":83,"threads_started":1}
I20260928 14:45:37.718806 24654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:37.718472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.67:55483)
0928 14:45:37.718675 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:37.718735 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":96,"threads_started":1}
I20260928 14:45:38.331137 24655 raft_consensus.cc:482] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:45:38.331280 24655 raft_consensus.cc:504] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b43bdfad8d3b4aa1bf499e9148871c0e" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 45783 } } peers { permanent_uuid: "fa74e6c10d9241628165ee829514df21" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33867 } } peers { permanent_uuid: "90ef635255234a8d8998cfa7951f1e32" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38363 } }
I20260928 14:45:38.331585 24655 leader_election.cc:288] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b43bdfad8d3b4aa1bf499e9148871c0e (127.9.33.67:45783), 90ef635255234a8d8998cfa7951f1e32 (127.9.33.66:38363)
W20260928 14:45:38.332113 24241 leader_election.cc:341] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 90ef635255234a8d8998cfa7951f1e32 (127.9.33.66:38363): Not found: Tablet not found: 7c7d62a1e9d64474a3a898bd887bf3b3
W20260928 14:45:38.332194 24243 leader_election.cc:341] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b43bdfad8d3b4aa1bf499e9148871c0e (127.9.33.67:45783): Not found: Tablet not found: 7c7d62a1e9d64474a3a898bd887bf3b3
I20260928 14:45:38.332239 24243 leader_election.cc:302] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa74e6c10d9241628165ee829514df21; no voters: 90ef635255234a8d8998cfa7951f1e32, b43bdfad8d3b4aa1bf499e9148871c0e
I20260928 14:45:38.332329 24655 raft_consensus.cc:2726] T 7c7d62a1e9d64474a3a898bd887bf3b3 P fa74e6c10d9241628165ee829514df21 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260928 14:45:38.534467 24656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:38.534140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.65:55097)
0928 14:45:38.534339 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:38.534399 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":91,"threads_started":1}
I20260928 14:45:38.600884 24657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:38.600577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.66:40769)
0928 14:45:38.600762 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:38.600817 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":83,"threads_started":1}
I20260928 14:45:38.719966 24658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:38.719478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.67:51325)
0928 14:45:38.719657 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:38.719884 (+ 227us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":77,"threads_started":1}
I20260928 14:45:39.535356 24659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:39.535113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.65:43675)
0928 14:45:39.535256 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:39.535306 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":66,"threads_started":1}
I20260928 14:45:39.601784 24660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:39.601514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.66:50785)
0928 14:45:39.601670 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:39.601727 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":70,"threads_started":1}
I20260928 14:45:39.720687 24661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:39.720466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.67:58673)
0928 14:45:39.720605 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:39.720640 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":62,"threads_started":1}
I20260928 14:45:40.536252 24662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:40.535926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.65:42701)
0928 14:45:40.536133 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:40.536200 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":81,"threads_started":1}
I20260928 14:45:40.602510 24663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:40.602281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.66:43815)
0928 14:45:40.602428 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:40.602465 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":80,"threads_started":1}
I20260928 14:45:40.721423 24664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:40.721184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.67:46777)
0928 14:45:40.721324 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:40.721366 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":72,"threads_started":1}
I20260928 14:45:41.537010 24665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:41.536770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.65:48097)
0928 14:45:41.536910 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:41.536959 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":66,"threads_started":1}
I20260928 14:45:41.603334 24666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:41.603102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.66:45383)
0928 14:45:41.603238 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:41.603284 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":62,"threads_started":1}
I20260928 14:45:41.722138 24667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:41.721913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.67:46533)
0928 14:45:41.722057 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:41.722088 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":80,"threads_started":1}
I20260928 14:45:42.537779 24668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:42.537533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.65:46461)
0928 14:45:42.537680 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:42.537729 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":77,"threads_started":1}
I20260928 14:45:42.604151 24669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:42.603901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.66:41051)
0928 14:45:42.604060 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:42.604101 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":94,"threads_started":1}
I20260928 14:45:42.722936 24670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:42.722682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.67:53681)
0928 14:45:42.722839 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:42.722884 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":81,"threads_started":1}
I20260928 14:45:43.538542 24671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:43.538317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.65:60861)
0928 14:45:43.538457 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:43.538491 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":71,"threads_started":1}
I20260928 14:45:43.605031 24672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:43.604766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.66:45257)
0928 14:45:43.604936 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:43.604981 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":92,"threads_started":1}
I20260928 14:45:43.723867 24673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:43.723558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.67:49531)
0928 14:45:43.723706 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:43.723811 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":84,"threads_started":1}
I20260928 14:45:44.539485 24674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:44.539131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.65:46421)
0928 14:45:44.539353 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:44.539405 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":61,"threads_started":1}
I20260928 14:45:44.605834 24675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:44.605624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.66:39255)
0928 14:45:44.605748 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:44.605788 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":64,"threads_started":1}
I20260928 14:45:44.724710 24676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:44.724478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46763 (local address 127.9.33.67:39013)
0928 14:45:44.724620 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:44.724662 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:46763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":79,"threads_started":1}
I20260928 14:45:44.726593 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 24196
I20260928 14:45:44.733179 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 24359
I20260928 14:45:44.737489 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 24492
2026-09-28T14:45:44Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18230 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:45:44 dist-test-slave-pv0w krb5kdc[24679](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:45:44 dist-test-slave-pv0w krb5kdc[24679](info): set up 2 sockets
Sep 28 14:45:44 dist-test-slave-pv0w krb5kdc[24679](info): commencing operation
krb5kdc: starting...
W20260928 14:45:45.780635 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:45:45 dist-test-slave-pv0w krb5kdc[24679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:45:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:45:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.33.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
I20260928 14:45:45.911979 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:37671
--webserver_interface=127.9.33.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:35845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:37671
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:45:45.995672 24692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:45:45.995946 24692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:45:45.995990 24692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:45:45.997205 24692 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260928 14:45:45.997279 24692 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260928 14:45:45.997300 24692 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260928 14:45:45.997320 24692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:45:45.997352 24692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:45:45.997395 24692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:45:45.997424 24692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:45:45.997454 24692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:45:45.999145 24692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:35845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:37671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:37671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:45:45.999440 24692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:45:45.999678 24692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:45:46.005065 24698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:46.005072 24697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:45:46.005363 24692 server_base.cc:1034] running on GCE node
W20260928 14:45:46.005072 24700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:45:46.005918 24692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:45:46.006187 24692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:45:46.007575 24692 hybrid_clock.cc:648] HybridClock initialized: now 1790606746007558 us; error 51 us; skew 500 ppm
Sep 28 14:45:46 dist-test-slave-pv0w krb5kdc[24679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606746, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:45:46.011384 24692 init.cc:377] Logged in from keytab as kudu/127.9.33.126@KRBTEST.COM (short username kudu)
I20260928 14:45:46.012056 24692 webserver.cc:460] Webserver started at http://127.9.33.126:38113/ using document root <none> and password file <none>
I20260928 14:45:46.012363 24692 fs_manager.cc:359] Metadata directory not provided
I20260928 14:45:46.012435 24692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:45:46.012645 24692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:45:46.014417 24692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "5959194892144cb0874ef1b45f8a628b"
format_stamp: "Formatted at 2026-09-28 14:45:46 on dist-test-slave-pv0w"
server_key: "fdea5ab4ddc9f7c2ad8ff985b067980e"
server_key_iv: "284d1a15ef73c20cc2603e8d5e0ad843"
server_key_version: "encryptionkey@0"
I20260928 14:45:46.014945 24692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "5959194892144cb0874ef1b45f8a628b"
format_stamp: "Formatted at 2026-09-28 14:45:46 on dist-test-slave-pv0w"
server_key: "fdea5ab4ddc9f7c2ad8ff985b067980e"
server_key_iv: "284d1a15ef73c20cc2603e8d5e0ad843"
server_key_version: "encryptionkey@0"
I20260928 14:45:46.016938 24692 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260928 14:45:46.017854 24707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:46.018069 24692 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:45:46.018185 24692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "5959194892144cb0874ef1b45f8a628b"
format_stamp: "Formatted at 2026-09-28 14:45:46 on dist-test-slave-pv0w"
server_key: "fdea5ab4ddc9f7c2ad8ff985b067980e"
server_key_iv: "284d1a15ef73c20cc2603e8d5e0ad843"
server_key_version: "encryptionkey@0"
I20260928 14:45:46.018287 24692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:45:46.038731 24692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:45:46.041339 24692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:45:46.041527 24692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:45:46.045722 24692 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:37671
I20260928 14:45:46.045754 24759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:37671 every 8 connection(s)
I20260928 14:45:46.046139 24692 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:45:46.047047 24760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:45:46.047899 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 24692
I20260928 14:45:46.048038 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:45:46.048223 9349 external_mini_cluster.cc:1428] Setting key d7c0709ef7e3dde887a5d3af9a4db224
I20260928 14:45:46.050143 24760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b: Bootstrap starting.
I20260928 14:45:46.050868 24760 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b: Neither blocks nor log segments found. Creating new log.
I20260928 14:45:46.051215 24760 log.cc:824] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b: Log is configured to *not* fsync() on all Append() calls
I20260928 14:45:46.052104 24760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b: No bootstrap required, opened a new log
I20260928 14:45:46.054508 24760 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5959194892144cb0874ef1b45f8a628b" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37671 } }
I20260928 14:45:46.054657 24760 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:45:46.054679 24760 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5959194892144cb0874ef1b45f8a628b, State: Initialized, Role: FOLLOWER
I20260928 14:45:46.054824 24760 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5959194892144cb0874ef1b45f8a628b" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37671 } }
I20260928 14:45:46.054912 24760 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:45:46.054958 24760 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:45:46.055011 24760 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:45:46.055626 24760 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5959194892144cb0874ef1b45f8a628b" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37671 } }
I20260928 14:45:46.055775 24760 leader_election.cc:302] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5959194892144cb0874ef1b45f8a628b; no voters:
I20260928 14:45:46.056005 24760 leader_election.cc:288] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [CANDIDATE]: Term 1 election: Requested vote from peers
Sep 28 14:45:46 dist-test-slave-pv0w krb5kdc[24679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:45:46.056043 24765 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:45:46.056164 24765 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [term 1 LEADER]: Becoming Leader. State: Replica: 5959194892144cb0874ef1b45f8a628b, State: Running, Role: LEADER
I20260928 14:45:46.056252 24765 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5959194892144cb0874ef1b45f8a628b" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37671 } }
I20260928 14:45:46.056327 24760 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:45:46.056595 24766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5959194892144cb0874ef1b45f8a628b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5959194892144cb0874ef1b45f8a628b" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37671 } } }
I20260928 14:45:46.056690 24766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [sys.catalog]: This master's current role is: LEADER
I20260928 14:45:46.056859 24767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5959194892144cb0874ef1b45f8a628b. Latest consensus state: current_term: 1 leader_uuid: "5959194892144cb0874ef1b45f8a628b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5959194892144cb0874ef1b45f8a628b" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37671 } } }
I20260928 14:45:46.056948 24767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [sys.catalog]: This master's current role is: LEADER
I20260928 14:45:46.057070 24769 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:45:46.057601 24769 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:45:46.059614 24769 catalog_manager.cc:1347] Generated new cluster ID: 1ec87a4d7c7e40a09ac7aa58277427c9
I20260928 14:45:46.059666 24769 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:45:46.060817 24763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:46.048945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54194 (local address 127.9.33.126:37671)
0928 14:45:46.049130 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:46.049141 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:46.049210 (+ 69us) server_negotiation.cc:407] Connection header received
0928 14:45:46.049825 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:46.049835 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:46.049904 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:46.050073 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:46.050793 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:46.052181 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:46.053013 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:46.053673 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:46.053828 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:46.056643 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:46.056648 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:46.056648 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:46.056661 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:46.058901 (+ 2240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:46.059348 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:46.059351 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:46.059351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:46.059468 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:46.059818 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:46.059819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:46.059819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:46.060147 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:46.060316 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:46.060446 (+ 130us) server_negotiation.cc:299] Negotiation successful
0928 14:45:46.060599 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":66,"threads_started":1}
I20260928 14:45:46.066700 24769 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:45:46.067219 24769 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:45:46.080346 24769 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b: Generated new TSK 0
I20260928 14:45:46.080590 24769 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.33.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab.
Entry for principal kudu/127.9.33.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab.
I20260928 14:45:46.115959 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.65:0
--local_ip_for_outbound_sockets=127.9.33.65
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:37671
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:35845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:45:46.206142 24786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:45:46.206409 24786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:45:46.206442 24786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:45:46.207958 24786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:45:46.208017 24786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:45:46.208074 24786 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.65
I20260928 14:45:46.209851 24786 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:35845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:37671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24786
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.65
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:45:46.210160 24786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:45:46.210439 24786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:45:46.216070 24794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:46.216085 24792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:46.216065 24791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:45:46.216109 24786 server_base.cc:1034] running on GCE node
I20260928 14:45:46.216496 24786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:45:46.216727 24786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:45:46.217896 24786 hybrid_clock.cc:648] HybridClock initialized: now 1790606746217879 us; error 29 us; skew 500 ppm
Sep 28 14:45:46 dist-test-slave-pv0w krb5kdc[24679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606746, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:45:46.220584 24786 init.cc:377] Logged in from keytab as kudu/127.9.33.65@KRBTEST.COM (short username kudu)
I20260928 14:45:46.221081 24786 webserver.cc:460] Webserver started at http://127.9.33.65:42967/ using document root <none> and password file <none>
I20260928 14:45:46.221391 24786 fs_manager.cc:359] Metadata directory not provided
I20260928 14:45:46.221446 24786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:45:46.221585 24786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:45:46.222849 24786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/data/instance:
uuid: "eafe320ad6134d8397a349b3e0f30e54"
format_stamp: "Formatted at 2026-09-28 14:45:46 on dist-test-slave-pv0w"
server_key: "54693ff559a2d17a617d166c70065be0"
server_key_iv: "1f946ada8ebc254bf91cd9d9eaa6fa84"
server_key_version: "encryptionkey@0"
I20260928 14:45:46.223237 24786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance:
uuid: "eafe320ad6134d8397a349b3e0f30e54"
format_stamp: "Formatted at 2026-09-28 14:45:46 on dist-test-slave-pv0w"
server_key: "54693ff559a2d17a617d166c70065be0"
server_key_iv: "1f946ada8ebc254bf91cd9d9eaa6fa84"
server_key_version: "encryptionkey@0"
I20260928 14:45:46.224716 24786 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:45:46.225564 24801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:46.225831 24786 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:45:46.225927 24786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/wal
uuid: "eafe320ad6134d8397a349b3e0f30e54"
format_stamp: "Formatted at 2026-09-28 14:45:46 on dist-test-slave-pv0w"
server_key: "54693ff559a2d17a617d166c70065be0"
server_key_iv: "1f946ada8ebc254bf91cd9d9eaa6fa84"
server_key_version: "encryptionkey@0"
I20260928 14:45:46.226001 24786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:45:46.235390 24786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:45:46.237788 24786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:45:46.237954 24786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:45:46.238230 24786 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:45:46.238581 24786 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:45:46.238641 24786 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:46.238687 24786 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:45:46.238718 24786 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:46.245253 24786 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.65:41211
I20260928 14:45:46.245296 24914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.65:41211 every 8 connection(s)
I20260928 14:45:46.245668 24786 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
I20260928 14:45:46.251353 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 24786
I20260928 14:45:46.251458 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance
I20260928 14:45:46.251701 9349 external_mini_cluster.cc:1428] Setting key 7e4315df7388fb504b573c465a2c71ca
Sep 28 14:45:46 dist-test-slave-pv0w krb5kdc[24679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606746, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:45:46.256763 24763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:46.246525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:48107 (local address 127.9.33.126:37671)
0928 14:45:46.246594 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:46.246596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:46.246820 (+ 224us) server_negotiation.cc:407] Connection header received
0928 14:45:46.247383 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:46.247384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:46.247398 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:46.247454 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:46.248591 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:46.249143 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:46.250155 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:46.250375 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:46.250455 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:46.253166 (+ 2711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:46.253168 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:46.253168 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:46.253176 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:46.254955 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:46.255476 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:46.255478 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:46.255478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:46.255544 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:46.255909 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:46.255910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:46.255911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:46.256038 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:46.256127 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:46.256559 (+ 432us) server_negotiation.cc:299] Negotiation successful
0928 14:45:46.256661 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 14:45:46.256990 24917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:46.246511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.65:48107)
0928 14:45:46.246690 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:46.246728 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:46.247272 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:46.247537 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:46.247540 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:46.248213 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:45:46.248464 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:46.248470 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:46.249247 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:46.249248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:46.250059 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:46.250061 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:46.250461 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:46.250463 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:46.250595 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:46.251170 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:45:46.251178 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:45:46.253049 (+ 1871us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:45:46.255053 (+ 2004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:46.255060 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:46.255062 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:46.255356 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:46.255624 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:46.255625 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:46.255626 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:46.255831 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:46.256116 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:45:46.256123 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:45:46.256405 (+ 282us) client_negotiation.cc:769] Sending connection context
0928 14:45:46.256523 (+ 118us) client_negotiation.cc:240] Negotiation successful
0928 14:45:46.256707 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":80,"threads_started":1}
I20260928 14:45:46.257548 24915 heartbeater.cc:347] Connected to a master server at 127.9.33.126:37671
I20260928 14:45:46.257668 24915 heartbeater.cc:464] Registering TS with master...
I20260928 14:45:46.257901 24915 heartbeater.cc:511] Master 127.9.33.126:37671 requested a full tablet report, sending...
I20260928 14:45:46.258523 24724 ts_manager.cc:194] Registered new tserver with Master: eafe320ad6134d8397a349b3e0f30e54 (127.9.33.65:41211)
I20260928 14:45:46.259300 24724 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.65@KRBTEST.COM'} at 127.9.33.65:48107
WARNING: no policy specified for kudu/127.9.33.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab.
Entry for principal kudu/127.9.33.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab.
I20260928 14:45:46.283049 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.66:0
--local_ip_for_outbound_sockets=127.9.33.66
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:37671
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:35845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:45:46.365566 24920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:45:46.365806 24920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:45:46.365833 24920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:45:46.367079 24920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:45:46.367133 24920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:45:46.367162 24920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.66
I20260928 14:45:46.368778 24920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:35845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:37671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.66
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:45:46.369083 24920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:45:46.369480 24920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:45:46.375034 24926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:46.375034 24925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:45:46.375152 24920 server_base.cc:1034] running on GCE node
W20260928 14:45:46.375056 24928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:45:46.375512 24920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:45:46.375845 24920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:45:46.377017 24920 hybrid_clock.cc:648] HybridClock initialized: now 1790606746377006 us; error 50 us; skew 500 ppm
Sep 28 14:45:46 dist-test-slave-pv0w krb5kdc[24679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606746, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:45:46.379779 24920 init.cc:377] Logged in from keytab as kudu/127.9.33.66@KRBTEST.COM (short username kudu)
I20260928 14:45:46.380245 24920 webserver.cc:460] Webserver started at http://127.9.33.66:35687/ using document root <none> and password file <none>
I20260928 14:45:46.380506 24920 fs_manager.cc:359] Metadata directory not provided
I20260928 14:45:46.380576 24920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:45:46.380734 24920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:45:46.381880 24920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/data/instance:
uuid: "41521aceb5454ef4977c9ffd7b683b6b"
format_stamp: "Formatted at 2026-09-28 14:45:46 on dist-test-slave-pv0w"
server_key: "c0465e976428b8a74d6866d2b5916d33"
server_key_iv: "cdaf2e85d120be8e711704a8a720e34d"
server_key_version: "encryptionkey@0"
I20260928 14:45:46.382262 24920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance:
uuid: "41521aceb5454ef4977c9ffd7b683b6b"
format_stamp: "Formatted at 2026-09-28 14:45:46 on dist-test-slave-pv0w"
server_key: "c0465e976428b8a74d6866d2b5916d33"
server_key_iv: "cdaf2e85d120be8e711704a8a720e34d"
server_key_version: "encryptionkey@0"
I20260928 14:45:46.383675 24920 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260928 14:45:46.384511 24935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:46.384684 24920 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:45:46.384797 24920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/wal
uuid: "41521aceb5454ef4977c9ffd7b683b6b"
format_stamp: "Formatted at 2026-09-28 14:45:46 on dist-test-slave-pv0w"
server_key: "c0465e976428b8a74d6866d2b5916d33"
server_key_iv: "cdaf2e85d120be8e711704a8a720e34d"
server_key_version: "encryptionkey@0"
I20260928 14:45:46.384876 24920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:45:46.402323 24920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:45:46.404915 24920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:45:46.405092 24920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:45:46.405354 24920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:45:46.405686 24920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:45:46.405746 24920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:46.405793 24920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:45:46.405819 24920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:46.411873 24920 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.66:46101
I20260928 14:45:46.411958 25048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.66:46101 every 8 connection(s)
I20260928 14:45:46.412536 24920 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
I20260928 14:45:46.418727 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 24920
I20260928 14:45:46.418864 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance
I20260928 14:45:46.419094 9349 external_mini_cluster.cc:1428] Setting key ea6c74bd4e02928d67424cf89fbb4719
Sep 28 14:45:46 dist-test-slave-pv0w krb5kdc[24679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606746, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:45:46.422963 24763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:46.413442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:57553 (local address 127.9.33.126:37671)
0928 14:45:46.413536 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:46.413538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:46.413698 (+ 160us) server_negotiation.cc:407] Connection header received
0928 14:45:46.414282 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:46.414284 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:46.414296 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:46.414336 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:46.415378 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:46.415984 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:46.417031 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:46.417232 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:46.417311 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:46.419867 (+ 2556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:46.419869 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:46.419869 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:46.419878 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:46.421472 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:46.421919 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:46.421920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:46.421920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:46.421985 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:46.422258 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:46.422259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:46.422259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:46.422401 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:46.422481 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:46.422779 (+ 298us) server_negotiation.cc:299] Negotiation successful
0928 14:45:46.422866 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260928 14:45:46.423187 25051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:46.413330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.66:57553)
0928 14:45:46.413550 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:46.413608 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:46.414151 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:46.414429 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:46.414432 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:46.415032 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:45:46.415261 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:46.415266 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:46.416088 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:46.416089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:46.416918 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:46.416921 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:46.417324 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:46.417325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:46.417455 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:46.418038 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:45:46.418047 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:45:46.419687 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:45:46.421571 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:46.421579 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:46.421582 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:46.421830 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:46.422056 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:46.422057 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:46.422057 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:46.422181 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:46.422471 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:45:46.422479 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:45:46.422631 (+ 152us) client_negotiation.cc:769] Sending connection context
0928 14:45:46.422742 (+ 111us) client_negotiation.cc:240] Negotiation successful
0928 14:45:46.422931 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":101,"threads_started":1}
I20260928 14:45:46.423888 25049 heartbeater.cc:347] Connected to a master server at 127.9.33.126:37671
I20260928 14:45:46.423981 25049 heartbeater.cc:464] Registering TS with master...
I20260928 14:45:46.424190 25049 heartbeater.cc:511] Master 127.9.33.126:37671 requested a full tablet report, sending...
I20260928 14:45:46.424621 24724 ts_manager.cc:194] Registered new tserver with Master: 41521aceb5454ef4977c9ffd7b683b6b (127.9.33.66:46101)
I20260928 14:45:46.425207 24724 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.66@KRBTEST.COM'} at 127.9.33.66:57553
WARNING: no policy specified for kudu/127.9.33.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab.
Entry for principal kudu/127.9.33.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab.
I20260928 14:45:46.448107 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.67:0
--local_ip_for_outbound_sockets=127.9.33.67
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:37671
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:35845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:45:46.535036 25054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:45:46.535271 25054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:45:46.535300 25054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:45:46.536602 25054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:45:46.536652 25054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:45:46.536681 25054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.67
I20260928 14:45:46.538197 25054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:35845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:37671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.67
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:45:46.538460 25054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:45:46.538734 25054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:45:46.544265 25060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:46.544302 25062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:45:46.544282 25059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:45:46.544548 25054 server_base.cc:1034] running on GCE node
I20260928 14:45:46.544785 25054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:45:46.545037 25054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:45:46.546181 25054 hybrid_clock.cc:648] HybridClock initialized: now 1790606746546171 us; error 26 us; skew 500 ppm
Sep 28 14:45:46 dist-test-slave-pv0w krb5kdc[24679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606746, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:45:46.548749 25054 init.cc:377] Logged in from keytab as kudu/127.9.33.67@KRBTEST.COM (short username kudu)
I20260928 14:45:46.549171 25054 webserver.cc:460] Webserver started at http://127.9.33.67:36613/ using document root <none> and password file <none>
I20260928 14:45:46.549384 25054 fs_manager.cc:359] Metadata directory not provided
I20260928 14:45:46.549427 25054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:45:46.549525 25054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:45:46.551002 25054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/data/instance:
uuid: "b86701327a744e368607ded503580579"
format_stamp: "Formatted at 2026-09-28 14:45:46 on dist-test-slave-pv0w"
server_key: "1745a03d9e9a59c21a864b52d9a5d2bd"
server_key_iv: "6296c5ef37a097596ab410c48d3b9a90"
server_key_version: "encryptionkey@0"
I20260928 14:45:46.551303 25054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance:
uuid: "b86701327a744e368607ded503580579"
format_stamp: "Formatted at 2026-09-28 14:45:46 on dist-test-slave-pv0w"
server_key: "1745a03d9e9a59c21a864b52d9a5d2bd"
server_key_iv: "6296c5ef37a097596ab410c48d3b9a90"
server_key_version: "encryptionkey@0"
I20260928 14:45:46.552731 25054 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:45:46.553574 25069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:46.553838 25054 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:45:46.553961 25054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/wal
uuid: "b86701327a744e368607ded503580579"
format_stamp: "Formatted at 2026-09-28 14:45:46 on dist-test-slave-pv0w"
server_key: "1745a03d9e9a59c21a864b52d9a5d2bd"
server_key_iv: "6296c5ef37a097596ab410c48d3b9a90"
server_key_version: "encryptionkey@0"
I20260928 14:45:46.554061 25054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:45:46.569757 25054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:45:46.572234 25054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:45:46.572408 25054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:45:46.572656 25054 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:45:46.572988 25054 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:45:46.573045 25054 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:46.573092 25054 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:45:46.573123 25054 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:45:46.579097 25054 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.67:34371
I20260928 14:45:46.579175 25182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.67:34371 every 8 connection(s)
I20260928 14:45:46.579546 25054 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
I20260928 14:45:46.583331 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 25054
I20260928 14:45:46.583428 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance
I20260928 14:45:46.583652 9349 external_mini_cluster.cc:1428] Setting key 3d6f8a17b4b073e830ac6178f38ff897
Sep 28 14:45:46 dist-test-slave-pv0w krb5kdc[24679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606746, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:45:46.590215 24763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:46.580430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:51123 (local address 127.9.33.126:37671)
0928 14:45:46.580486 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:46.580489 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:46.580735 (+ 246us) server_negotiation.cc:407] Connection header received
0928 14:45:46.581396 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:46.581398 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:46.581411 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:46.581462 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:46.582474 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:46.582990 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:46.584489 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:46.584769 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:46.584851 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:46.587196 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:46.587198 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:46.587198 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:46.587208 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:46.588735 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:46.589208 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:46.589210 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:46.589211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:46.589273 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:46.589541 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:46.589542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:46.589543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:46.589671 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:46.589744 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:46.590033 (+ 289us) server_negotiation.cc:299] Negotiation successful
0928 14:45:46.590116 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 14:45:46.590451 25185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:46.580431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.67:51123)
0928 14:45:46.580602 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:46.580640 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:46.581265 (+ 625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:46.581541 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:46.581545 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:46.582143 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:45:46.582351 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:46.582355 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:46.583093 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:46.583094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:46.584407 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:46.584410 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:46.584872 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:46.584873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:46.585014 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:46.585593 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:45:46.585602 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:45:46.587087 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:45:46.588852 (+ 1765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:46.588860 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:46.588862 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:46.589107 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:46.589346 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:45:46.589347 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:45:46.589348 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:45:46.589471 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:45:46.589744 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:45:46.589753 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:45:46.589900 (+ 147us) client_negotiation.cc:769] Sending connection context
0928 14:45:46.590011 (+ 111us) client_negotiation.cc:240] Negotiation successful
0928 14:45:46.590185 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":76,"threads_started":1}
I20260928 14:45:46.591064 25183 heartbeater.cc:347] Connected to a master server at 127.9.33.126:37671
I20260928 14:45:46.591179 25183 heartbeater.cc:464] Registering TS with master...
I20260928 14:45:46.591398 25183 heartbeater.cc:511] Master 127.9.33.126:37671 requested a full tablet report, sending...
I20260928 14:45:46.591879 24724 ts_manager.cc:194] Registered new tserver with Master: b86701327a744e368607ded503580579 (127.9.33.67:34371)
I20260928 14:45:46.592264 24724 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.67@KRBTEST.COM'} at 127.9.33.67:51123
I20260928 14:45:46.595589 9349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:45:46.603318 24763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:46.597081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54204 (local address 127.9.33.126:37671)
0928 14:45:46.597138 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:46.597140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:46.597243 (+ 103us) server_negotiation.cc:407] Connection header received
0928 14:45:46.597309 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:46.597309 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:46.597319 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:46.597358 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:45:46.597901 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:46.598414 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:46.599160 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:46.599358 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:46.599416 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:46.600669 (+ 1253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:45:46.600670 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:45:46.600670 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:45:46.600677 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:45:46.602199 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:46.602515 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:46.602516 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:46.602516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:46.602566 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:45:46.602810 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:45:46.602812 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:45:46.602812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:45:46.602976 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:45:46.603068 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:46.603170 (+ 102us) server_negotiation.cc:299] Negotiation successful
0928 14:45:46.603252 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260928 14:45:46.605157 24724 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54204:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:45:46.606930 24724 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:45:46.614209 25197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:46.609700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50926 (local address 127.9.33.66:46101)
0928 14:45:46.609924 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:46.609930 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:46.609940 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:45:46.609981 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:46.609984 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:46.609998 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:46.610108 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:45:46.610370 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:46.611531 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:46.613061 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:46.613766 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:46.613854 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:46.613918 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:46.614065 (+ 147us) server_negotiation.cc:299] Negotiation successful
0928 14:45:46.614112 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":95,"threads_started":1}
I20260928 14:45:46.614217 25194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:46.609572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:46101 (local address 127.0.0.1:50926)
0928 14:45:46.609742 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:46.609800 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:46.609871 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:46.610109 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:46.610112 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:46.610115 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:45:46.610196 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:46.610198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:46.611645 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:46.611646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:46.612975 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:46.612978 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:46.613869 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:46.613870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:46.613934 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:46.613946 (+ 12us) client_negotiation.cc:769] Sending connection context
0928 14:45:46.614048 (+ 102us) client_negotiation.cc:240] Negotiation successful
0928 14:45:46.614108 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":38,"thread_start_us":57,"threads_started":1}
I20260928 14:45:46.614351 25198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:46.609833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40618 (local address 127.9.33.65:41211)
0928 14:45:46.610015 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:46.610020 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:46.610474 (+ 454us) server_negotiation.cc:407] Connection header received
0928 14:45:46.610629 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:46.610632 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:46.610650 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:46.610731 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:45:46.610846 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:46.611887 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:46.613282 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:46.613956 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:46.614021 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:46.614073 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:46.614210 (+ 137us) server_negotiation.cc:299] Negotiation successful
0928 14:45:46.614263 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":107,"threads_started":1}
I20260928 14:45:46.614638 25196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:46.609573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:41211 (local address 127.0.0.1:40618)
0928 14:45:46.610404 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:46.610414 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:46.610505 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:46.610739 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:46.610740 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:46.610742 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:45:46.610788 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:46.610789 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:46.611991 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:46.611992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:46.613181 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:46.613184 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:46.614040 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:46.614041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:46.614100 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:46.614103 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:45:46.614490 (+ 387us) client_negotiation.cc:240] Negotiation successful
0928 14:45:46.614538 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":808,"mutex_wait_us":2,"thread_start_us":91,"threads_started":1}
I20260928 14:45:46.615330 25195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:46.609573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:34371 (local address 127.0.0.1:46716)
0928 14:45:46.610203 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:46.610214 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:46.610307 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:46.610575 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:46.610576 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:46.610578 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:45:46.610622 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:46.610623 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:46.611620 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:46.611621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:46.613181 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:46.613184 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:46.615066 (+ 1882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:46.615067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:46.615132 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:46.615136 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:45:46.615197 (+ 61us) client_negotiation.cc:240] Negotiation successful
0928 14:45:46.615254 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"mutex_wait_us":43,"thread_start_us":96,"threads_started":1}
I20260928 14:45:46.615383 25199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:46.610044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46716 (local address 127.9.33.67:34371)
0928 14:45:46.610216 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:46.610220 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:46.610309 (+ 89us) server_negotiation.cc:407] Connection header received
0928 14:45:46.610386 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:46.610389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:46.610406 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:46.610487 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:45:46.610681 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:46.611504 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:46.614111 (+ 2607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:46.614900 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:46.614971 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:46.615029 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:46.615245 (+ 216us) server_negotiation.cc:299] Negotiation successful
0928 14:45:46.615298 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":107,"threads_started":1}
I20260928 14:45:46.615474 24849 tablet_service.cc:1467] Processing CreateTablet for tablet 5e91bed5a5024fe6a693b14a85eec62f (DEFAULT_TABLE table=test-table [id=9bc042ced5c048de9f70411bee32d7cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:45:46.615532 24983 tablet_service.cc:1467] Processing CreateTablet for tablet 5e91bed5a5024fe6a693b14a85eec62f (DEFAULT_TABLE table=test-table [id=9bc042ced5c048de9f70411bee32d7cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:45:46.615866 24849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e91bed5a5024fe6a693b14a85eec62f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:45:46.615883 24983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e91bed5a5024fe6a693b14a85eec62f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:45:46.616381 25117 tablet_service.cc:1467] Processing CreateTablet for tablet 5e91bed5a5024fe6a693b14a85eec62f (DEFAULT_TABLE table=test-table [id=9bc042ced5c048de9f70411bee32d7cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:45:46.616696 25117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e91bed5a5024fe6a693b14a85eec62f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:45:46.618813 25200 tablet_bootstrap.cc:492] T 5e91bed5a5024fe6a693b14a85eec62f P eafe320ad6134d8397a349b3e0f30e54: Bootstrap starting.
I20260928 14:45:46.619249 25201 tablet_bootstrap.cc:492] T 5e91bed5a5024fe6a693b14a85eec62f P 41521aceb5454ef4977c9ffd7b683b6b: Bootstrap starting.
I20260928 14:45:46.619659 25200 tablet_bootstrap.cc:654] T 5e91bed5a5024fe6a693b14a85eec62f P eafe320ad6134d8397a349b3e0f30e54: Neither blocks nor log segments found. Creating new log.
I20260928 14:45:46.619985 25201 tablet_bootstrap.cc:654] T 5e91bed5a5024fe6a693b14a85eec62f P 41521aceb5454ef4977c9ffd7b683b6b: Neither blocks nor log segments found. Creating new log.
I20260928 14:45:46.620113 25202 tablet_bootstrap.cc:492] T 5e91bed5a5024fe6a693b14a85eec62f P b86701327a744e368607ded503580579: Bootstrap starting.
I20260928 14:45:46.620401 25200 log.cc:824] T 5e91bed5a5024fe6a693b14a85eec62f P eafe320ad6134d8397a349b3e0f30e54: Log is configured to *not* fsync() on all Append() calls
I20260928 14:45:46.620651 25201 log.cc:824] T 5e91bed5a5024fe6a693b14a85eec62f P 41521aceb5454ef4977c9ffd7b683b6b: Log is configured to *not* fsync() on all Append() calls
I20260928 14:45:46.620805 25202 tablet_bootstrap.cc:654] T 5e91bed5a5024fe6a693b14a85eec62f P b86701327a744e368607ded503580579: Neither blocks nor log segments found. Creating new log.
I20260928 14:45:46.621284 25200 tablet_bootstrap.cc:492] T 5e91bed5a5024fe6a693b14a85eec62f P eafe320ad6134d8397a349b3e0f30e54: No bootstrap required, opened a new log
I20260928 14:45:46.621378 25200 ts_tablet_manager.cc:1397] T 5e91bed5a5024fe6a693b14a85eec62f P eafe320ad6134d8397a349b3e0f30e54: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260928 14:45:46.621490 25201 tablet_bootstrap.cc:492] T 5e91bed5a5024fe6a693b14a85eec62f P 41521aceb5454ef4977c9ffd7b683b6b: No bootstrap required, opened a new log
I20260928 14:45:46.621469 25202 log.cc:824] T 5e91bed5a5024fe6a693b14a85eec62f P b86701327a744e368607ded503580579: Log is configured to *not* fsync() on all Append() calls
I20260928 14:45:46.621556 25201 ts_tablet_manager.cc:1397] T 5e91bed5a5024fe6a693b14a85eec62f P 41521aceb5454ef4977c9ffd7b683b6b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:45:46.622263 25202 tablet_bootstrap.cc:492] T 5e91bed5a5024fe6a693b14a85eec62f P b86701327a744e368607ded503580579: No bootstrap required, opened a new log
I20260928 14:45:46.622336 25202 ts_tablet_manager.cc:1397] T 5e91bed5a5024fe6a693b14a85eec62f P b86701327a744e368607ded503580579: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:45:46.623324 25200 raft_consensus.cc:348] T 5e91bed5a5024fe6a693b14a85eec62f P eafe320ad6134d8397a349b3e0f30e54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eafe320ad6134d8397a349b3e0f30e54" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41211 } } peers { permanent_uuid: "41521aceb5454ef4977c9ffd7b683b6b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 46101 } } peers { permanent_uuid: "b86701327a744e368607ded503580579" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34371 } }
I20260928 14:45:46.623508 25200 raft_consensus.cc:374] T 5e91bed5a5024fe6a693b14a85eec62f P eafe320ad6134d8397a349b3e0f30e54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:45:46.623557 25200 raft_consensus.cc:729] T 5e91bed5a5024fe6a693b14a85eec62f P eafe320ad6134d8397a349b3e0f30e54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eafe320ad6134d8397a349b3e0f30e54, State: Initialized, Role: FOLLOWER
I20260928 14:45:46.623700 25200 consensus_queue.cc:260] T 5e91bed5a5024fe6a693b14a85eec62f P eafe320ad6134d8397a349b3e0f30e54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eafe320ad6134d8397a349b3e0f30e54" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41211 } } peers { permanent_uuid: "41521aceb5454ef4977c9ffd7b683b6b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 46101 } } peers { permanent_uuid: "b86701327a744e368607ded503580579" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34371 } }
I20260928 14:45:46.623605 25201 raft_consensus.cc:348] T 5e91bed5a5024fe6a693b14a85eec62f P 41521aceb5454ef4977c9ffd7b683b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eafe320ad6134d8397a349b3e0f30e54" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41211 } } peers { permanent_uuid: "41521aceb5454ef4977c9ffd7b683b6b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 46101 } } peers { permanent_uuid: "b86701327a744e368607ded503580579" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34371 } }
I20260928 14:45:46.624061 25200 ts_tablet_manager.cc:1428] T 5e91bed5a5024fe6a693b14a85eec62f P eafe320ad6134d8397a349b3e0f30e54: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:45:46.624070 25201 raft_consensus.cc:374] T 5e91bed5a5024fe6a693b14a85eec62f P 41521aceb5454ef4977c9ffd7b683b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:45:46.624121 25201 raft_consensus.cc:729] T 5e91bed5a5024fe6a693b14a85eec62f P 41521aceb5454ef4977c9ffd7b683b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41521aceb5454ef4977c9ffd7b683b6b, State: Initialized, Role: FOLLOWER
I20260928 14:45:46.624210 24915 heartbeater.cc:503] Master 127.9.33.126:37671 was elected leader, sending a full tablet report...
I20260928 14:45:46.624239 25201 consensus_queue.cc:260] T 5e91bed5a5024fe6a693b14a85eec62f P 41521aceb5454ef4977c9ffd7b683b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eafe320ad6134d8397a349b3e0f30e54" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41211 } } peers { permanent_uuid: "41521aceb5454ef4977c9ffd7b683b6b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 46101 } } peers { permanent_uuid: "b86701327a744e368607ded503580579" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34371 } }
I20260928 14:45:46.624383 25202 raft_consensus.cc:348] T 5e91bed5a5024fe6a693b14a85eec62f P b86701327a744e368607ded503580579 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eafe320ad6134d8397a349b3e0f30e54" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41211 } } peers { permanent_uuid: "41521aceb5454ef4977c9ffd7b683b6b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 46101 } } peers { permanent_uuid: "b86701327a744e368607ded503580579" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34371 } }
I20260928 14:45:46.624542 25201 ts_tablet_manager.cc:1428] T 5e91bed5a5024fe6a693b14a85eec62f P 41521aceb5454ef4977c9ffd7b683b6b: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260928 14:45:46.624557 25202 raft_consensus.cc:374] T 5e91bed5a5024fe6a693b14a85eec62f P b86701327a744e368607ded503580579 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:45:46.624559 25049 heartbeater.cc:503] Master 127.9.33.126:37671 was elected leader, sending a full tablet report...
I20260928 14:45:46.624593 25202 raft_consensus.cc:729] T 5e91bed5a5024fe6a693b14a85eec62f P b86701327a744e368607ded503580579 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b86701327a744e368607ded503580579, State: Initialized, Role: FOLLOWER
I20260928 14:45:46.624703 25202 consensus_queue.cc:260] T 5e91bed5a5024fe6a693b14a85eec62f P b86701327a744e368607ded503580579 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eafe320ad6134d8397a349b3e0f30e54" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41211 } } peers { permanent_uuid: "41521aceb5454ef4977c9ffd7b683b6b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 46101 } } peers { permanent_uuid: "b86701327a744e368607ded503580579" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34371 } }
I20260928 14:45:46.624994 25202 ts_tablet_manager.cc:1428] T 5e91bed5a5024fe6a693b14a85eec62f P b86701327a744e368607ded503580579: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:45:46.624994 25183 heartbeater.cc:503] Master 127.9.33.126:37671 was elected leader, sending a full tablet report...
W20260928 14:45:46.663137 25050 tablet.cc:2369] T 5e91bed5a5024fe6a693b14a85eec62f P 41521aceb5454ef4977c9ffd7b683b6b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:45:46.746461 24916 tablet.cc:2369] T 5e91bed5a5024fe6a693b14a85eec62f P eafe320ad6134d8397a349b3e0f30e54: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:45:46.779506 25208 raft_consensus.cc:482] T 5e91bed5a5024fe6a693b14a85eec62f P b86701327a744e368607ded503580579 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:45:46.779639 25208 raft_consensus.cc:504] T 5e91bed5a5024fe6a693b14a85eec62f P b86701327a744e368607ded503580579 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eafe320ad6134d8397a349b3e0f30e54" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41211 } } peers { permanent_uuid: "41521aceb5454ef4977c9ffd7b683b6b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 46101 } } peers { permanent_uuid: "b86701327a744e368607ded503580579" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34371 } }
I20260928 14:45:46.780066 25208 leader_election.cc:288] T 5e91bed5a5024fe6a693b14a85eec62f P b86701327a744e368607ded503580579 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eafe320ad6134d8397a349b3e0f30e54 (127.9.33.65:41211), 41521aceb5454ef4977c9ffd7b683b6b (127.9.33.66:46101)
I20260928 14:45:46.783270 25197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:46.780445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:43297 (local address 127.9.33.66:46101)
0928 14:45:46.780500 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:46.780501 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:46.780512 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:45:46.780568 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:46.780569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:46.780588 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:46.780645 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:45:46.780831 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:46.781427 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:46.782486 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:46.782877 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:46.782976 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:46.782991 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:46.783160 (+ 169us) server_negotiation.cc:299] Negotiation successful
0928 14:45:46.783202 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260928 14:45:46.783272 25185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:46.780330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:46101 (local address 127.9.33.67:43297)
0928 14:45:46.780420 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:46.780434 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:46.780505 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:46.780660 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:46.780661 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:46.780663 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:45:46.780749 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:46.780751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:46.781529 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:46.781530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:46.782393 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:46.782396 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:46.782981 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:46.782982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:46.783067 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:46.783070 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:45:46.783133 (+ 63us) client_negotiation.cc:240] Negotiation successful
0928 14:45:46.783188 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"mutex_wait_us":3}
I20260928 14:45:46.783362 25198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:46.780433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:51863 (local address 127.9.33.65:41211)
0928 14:45:46.780483 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:46.780486 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:46.780609 (+ 123us) server_negotiation.cc:407] Connection header received
0928 14:45:46.780709 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:46.780710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:46.780729 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:46.780804 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:45:46.780979 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:46.781506 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:46.782594 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:46.782973 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:46.783056 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:46.783066 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:46.783218 (+ 152us) server_negotiation.cc:299] Negotiation successful
0928 14:45:46.783263 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260928 14:45:46.783417 25209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:46.780330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:41211 (local address 127.9.33.67:51863)
0928 14:45:46.780535 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:46.780544 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 14:45:46.780624 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:45:46.780808 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:45:46.780809 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:45:46.780811 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:45:46.780897 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:46.780898 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:46.781610 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:46.781611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:46.782495 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:45:46.782498 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:45:46.783063 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:45:46.783064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:45:46.783139 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:46.783145 (+ 6us) client_negotiation.cc:769] Sending connection context
0928 14:45:46.783213 (+ 68us) client_negotiation.cc:240] Negotiation successful
0928 14:45:46.783256 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":24,"thread_start_us":106,"threads_started":1}
I20260928 14:45:46.783834 24869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e91bed5a5024fe6a693b14a85eec62f" candidate_uuid: "b86701327a744e368607ded503580579" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eafe320ad6134d8397a349b3e0f30e54" is_pre_election: true
I20260928 14:45:46.784063 24869 raft_consensus.cc:2454] T 5e91bed5a5024fe6a693b14a85eec62f P eafe320ad6134d8397a349b3e0f30e54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b86701327a744e368607ded503580579 in term 0.
I20260928 14:45:46.784165 25003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e91bed5a5024fe6a693b14a85eec62f" candidate_uuid: "b86701327a744e368607ded503580579" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41521aceb5454ef4977c9ffd7b683b6b" is_pre_election: true
I20260928 14:45:46.784351 25003 raft_consensus.cc:2454] T 5e91bed5a5024fe6a693b14a85eec62f P 41521aceb5454ef4977c9ffd7b683b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b86701327a744e368607ded503580579 in term 0.
I20260928 14:45:46.784354 25073 leader_election.cc:302] T 5e91bed5a5024fe6a693b14a85eec62f P b86701327a744e368607ded503580579 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b86701327a744e368607ded503580579, eafe320ad6134d8397a349b3e0f30e54; no voters:
I20260928 14:45:46.784590 25208 raft_consensus.cc:2781] T 5e91bed5a5024fe6a693b14a85eec62f P b86701327a744e368607ded503580579 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:45:46.784651 25208 raft_consensus.cc:482] T 5e91bed5a5024fe6a693b14a85eec62f P b86701327a744e368607ded503580579 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:45:46.784695 25208 raft_consensus.cc:3037] T 5e91bed5a5024fe6a693b14a85eec62f P b86701327a744e368607ded503580579 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:45:46.785488 25208 raft_consensus.cc:504] T 5e91bed5a5024fe6a693b14a85eec62f P b86701327a744e368607ded503580579 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eafe320ad6134d8397a349b3e0f30e54" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41211 } } peers { permanent_uuid: "41521aceb5454ef4977c9ffd7b683b6b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 46101 } } peers { permanent_uuid: "b86701327a744e368607ded503580579" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34371 } }
I20260928 14:45:46.785656 25208 leader_election.cc:288] T 5e91bed5a5024fe6a693b14a85eec62f P b86701327a744e368607ded503580579 [CANDIDATE]: Term 1 election: Requested vote from peers eafe320ad6134d8397a349b3e0f30e54 (127.9.33.65:41211), 41521aceb5454ef4977c9ffd7b683b6b (127.9.33.66:46101)
I20260928 14:45:46.785846 25003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e91bed5a5024fe6a693b14a85eec62f" candidate_uuid: "b86701327a744e368607ded503580579" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41521aceb5454ef4977c9ffd7b683b6b"
I20260928 14:45:46.785898 24869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e91bed5a5024fe6a693b14a85eec62f" candidate_uuid: "b86701327a744e368607ded503580579" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eafe320ad6134d8397a349b3e0f30e54"
I20260928 14:45:46.785935 25003 raft_consensus.cc:3037] T 5e91bed5a5024fe6a693b14a85eec62f P 41521aceb5454ef4977c9ffd7b683b6b [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:45:46.786024 24869 raft_consensus.cc:3037] T 5e91bed5a5024fe6a693b14a85eec62f P eafe320ad6134d8397a349b3e0f30e54 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:45:46.786748 25003 raft_consensus.cc:2454] T 5e91bed5a5024fe6a693b14a85eec62f P 41521aceb5454ef4977c9ffd7b683b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b86701327a744e368607ded503580579 in term 1.
I20260928 14:45:46.786866 24869 raft_consensus.cc:2454] T 5e91bed5a5024fe6a693b14a85eec62f P eafe320ad6134d8397a349b3e0f30e54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b86701327a744e368607ded503580579 in term 1.
I20260928 14:45:46.786936 25071 leader_election.cc:302] T 5e91bed5a5024fe6a693b14a85eec62f P b86701327a744e368607ded503580579 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41521aceb5454ef4977c9ffd7b683b6b, b86701327a744e368607ded503580579; no voters:
I20260928 14:45:46.787057 25208 raft_consensus.cc:2781] T 5e91bed5a5024fe6a693b14a85eec62f P b86701327a744e368607ded503580579 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:45:46.787241 25208 raft_consensus.cc:686] T 5e91bed5a5024fe6a693b14a85eec62f P b86701327a744e368607ded503580579 [term 1 LEADER]: Becoming Leader. State: Replica: b86701327a744e368607ded503580579, State: Running, Role: LEADER
I20260928 14:45:46.787518 25208 consensus_queue.cc:237] T 5e91bed5a5024fe6a693b14a85eec62f P b86701327a744e368607ded503580579 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eafe320ad6134d8397a349b3e0f30e54" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41211 } } peers { permanent_uuid: "41521aceb5454ef4977c9ffd7b683b6b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 46101 } } peers { permanent_uuid: "b86701327a744e368607ded503580579" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34371 } }
I20260928 14:45:46.788470 24722 catalog_manager.cc:5495] T 5e91bed5a5024fe6a693b14a85eec62f P b86701327a744e368607ded503580579 reported cstate change: term changed from 0 to 1, leader changed from <none> to b86701327a744e368607ded503580579 (127.9.33.67). New cstate: current_term: 1 leader_uuid: "b86701327a744e368607ded503580579" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eafe320ad6134d8397a349b3e0f30e54" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41211 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41521aceb5454ef4977c9ffd7b683b6b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 46101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b86701327a744e368607ded503580579" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34371 } health_report { overall_health: HEALTHY } } }
I20260928 14:45:46.802656 25199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:46.800087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46728 (local address 127.9.33.67:34371)
0928 14:45:46.800170 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0928 14:45:46.800172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:45:46.800180 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:45:46.800224 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:45:46.800224 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:45:46.800286 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:45:46.800372 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:45:46.800507 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:46.800991 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:46.801794 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:45:46.802003 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:45:46.802085 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:45:46.802283 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:45:46.802392 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:45:46.802494 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0928 14:45:46.802513 (+ 19us) server_negotiation.cc:299] Negotiation successful
0928 14:45:46.802535 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260928 14:45:46.804687 24869 raft_consensus.cc:1261] T 5e91bed5a5024fe6a693b14a85eec62f P eafe320ad6134d8397a349b3e0f30e54 [term 1 FOLLOWER]: Refusing update from remote peer b86701327a744e368607ded503580579: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:45:46.804693 25003 raft_consensus.cc:1261] T 5e91bed5a5024fe6a693b14a85eec62f P 41521aceb5454ef4977c9ffd7b683b6b [term 1 FOLLOWER]: Refusing update from remote peer b86701327a744e368607ded503580579: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:45:46.805109 25210 consensus_queue.cc:1035] T 5e91bed5a5024fe6a693b14a85eec62f P b86701327a744e368607ded503580579 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41521aceb5454ef4977c9ffd7b683b6b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 46101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:45:46.805258 25208 consensus_queue.cc:1035] T 5e91bed5a5024fe6a693b14a85eec62f P b86701327a744e368607ded503580579 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eafe320ad6134d8397a349b3e0f30e54" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:45:46.810724 24722 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:45:46.810869 24722 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:45:46.811839 24722 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b: Sending DeleteTablet for 3 replicas of tablet 5e91bed5a5024fe6a693b14a85eec62f
I20260928 14:45:46.812251 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 24786
I20260928 14:45:46.812219 24849 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e91bed5a5024fe6a693b14a85eec62f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:45:46 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:40618
I20260928 14:45:46.812235 25117 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e91bed5a5024fe6a693b14a85eec62f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:45:46 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:46716
I20260928 14:45:46.812364 24983 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e91bed5a5024fe6a693b14a85eec62f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:45:46 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:50926
I20260928 14:45:46.812860 25224 tablet_replica.cc:330] T 5e91bed5a5024fe6a693b14a85eec62f P 41521aceb5454ef4977c9ffd7b683b6b: stopping tablet replica
I20260928 14:45:46.812995 25224 raft_consensus.cc:2229] T 5e91bed5a5024fe6a693b14a85eec62f P 41521aceb5454ef4977c9ffd7b683b6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:45:46.813122 25224 raft_consensus.cc:2258] T 5e91bed5a5024fe6a693b14a85eec62f P 41521aceb5454ef4977c9ffd7b683b6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:45:46.813570 25224 ts_tablet_manager.cc:1905] T 5e91bed5a5024fe6a693b14a85eec62f P 41521aceb5454ef4977c9ffd7b683b6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:45:46.815150 25224 ts_tablet_manager.cc:1918] T 5e91bed5a5024fe6a693b14a85eec62f P 41521aceb5454ef4977c9ffd7b683b6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:45:46.815286 25224 log.cc:1196] T 5e91bed5a5024fe6a693b14a85eec62f P 41521aceb5454ef4977c9ffd7b683b6b: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/wal/wals/5e91bed5a5024fe6a693b14a85eec62f
I20260928 14:45:46.815601 25224 ts_tablet_manager.cc:1939] T 5e91bed5a5024fe6a693b14a85eec62f P 41521aceb5454ef4977c9ffd7b683b6b: Deleting consensus metadata
I20260928 14:45:46.816124 24709 catalog_manager.cc:4841] TS 41521aceb5454ef4977c9ffd7b683b6b (127.9.33.66:46101): tablet 5e91bed5a5024fe6a693b14a85eec62f (table test-table [id=9bc042ced5c048de9f70411bee32d7cb]) successfully deleted
W20260928 14:45:46.818909 24711 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.33.65:41211 (error 108)
I20260928 14:45:46.819084 25223 tablet_replica.cc:330] T 5e91bed5a5024fe6a693b14a85eec62f P b86701327a744e368607ded503580579: stopping tablet replica
I20260928 14:45:46.819255 25223 raft_consensus.cc:2229] T 5e91bed5a5024fe6a693b14a85eec62f P b86701327a744e368607ded503580579 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:45:46.819384 25223 raft_consensus.cc:2258] T 5e91bed5a5024fe6a693b14a85eec62f P b86701327a744e368607ded503580579 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:45:46.819625 25195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:45:46.819490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:41211 (local address 127.0.0.1:40624)
0928 14:45:46.819538 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0928 14:45:46.819584 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.65:41211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260928 14:45:46.819686 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 24920
W20260928 14:45:46.819796 24711 catalog_manager.cc:4584] TS eafe320ad6134d8397a349b3e0f30e54 (127.9.33.65:41211): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5e91bed5a5024fe6a693b14a85eec62f: Network error: Client connection negotiation failed: client connection to 127.9.33.65:41211: connect: Connection refused (error 111)
I20260928 14:45:46.820803 25223 ts_tablet_manager.cc:1905] T 5e91bed5a5024fe6a693b14a85eec62f P b86701327a744e368607ded503580579: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:45:46.822755 25223 ts_tablet_manager.cc:1918] T 5e91bed5a5024fe6a693b14a85eec62f P b86701327a744e368607ded503580579: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:45:46.822844 25223 log.cc:1196] T 5e91bed5a5024fe6a693b14a85eec62f P b86701327a744e368607ded503580579: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/wal/wals/5e91bed5a5024fe6a693b14a85eec62f
I20260928 14:45:46.823117 25223 ts_tablet_manager.cc:1939] T 5e91bed5a5024fe6a693b14a85eec62f P b86701327a744e368607ded503580579: Deleting consensus metadata
I20260928 14:45:46.823590 24708 catalog_manager.cc:4841] TS b86701327a744e368607ded503580579 (127.9.33.67:34371): tablet 5e91bed5a5024fe6a693b14a85eec62f (table test-table [id=9bc042ced5c048de9f70411bee32d7cb]) successfully deleted
I20260928 14:45:46.827603 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 25054
I20260928 14:45:46.832990 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 24692
I20260928 14:46:06.840461 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:37671
--webserver_interface=127.9.33.126
--webserver_port=38113
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:35845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:37671
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:46:06.934525 25226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:46:06.934923 25226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:46:06.934979 25226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:46:06.936551 25226 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260928 14:46:06.936640 25226 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260928 14:46:06.936663 25226 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260928 14:46:06.936684 25226 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:46:06.936719 25226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:46:06.936745 25226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:46:06.936774 25226 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:46:06.936801 25226 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:46:06.938721 25226 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:35845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:37671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:37671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=38113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:46:06.939088 25226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:46:06.939388 25226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 14:46:06.950457 25226 server_base.cc:1034] running on GCE node
W20260928 14:46:06.950414 25235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:06.950407 25232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:06.950407 25233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:06.951109 25226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:46:06.951574 25226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:46:06.952802 25226 hybrid_clock.cc:648] HybridClock initialized: now 1790606766952771 us; error 48 us; skew 500 ppm
Sep 28 14:46:06 dist-test-slave-pv0w krb5kdc[24679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606766, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:46:06.956305 25226 init.cc:377] Logged in from keytab as kudu/127.9.33.126@KRBTEST.COM (short username kudu)
I20260928 14:46:06.956952 25226 webserver.cc:460] Webserver started at http://127.9.33.126:38113/ using document root <none> and password file <none>
I20260928 14:46:06.957263 25226 fs_manager.cc:359] Metadata directory not provided
I20260928 14:46:06.957366 25226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:46:06.959070 25226 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260928 14:46:06.959936 25242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:06.960157 25226 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:46:06.960274 25226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "5959194892144cb0874ef1b45f8a628b"
format_stamp: "Formatted at 2026-09-28 14:45:46 on dist-test-slave-pv0w"
server_key: "fdea5ab4ddc9f7c2ad8ff985b067980e"
server_key_iv: "284d1a15ef73c20cc2603e8d5e0ad843"
server_key_version: "encryptionkey@0"
I20260928 14:46:06.960830 25226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:46:06.983085 25226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:46:06.985821 25226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:46:06.985961 25226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:46:06.991026 25226 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:37671
I20260928 14:46:06.991119 25294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:37671 every 8 connection(s)
I20260928 14:46:06.991602 25226 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:46:06.993491 25295 sys_catalog.cc:263] Verifying existing consensus state
I20260928 14:46:06.994287 25295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b: Bootstrap starting.
I20260928 14:46:06.997936 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 25226
I20260928 14:46:06.998373 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.65:41211
--local_ip_for_outbound_sockets=127.9.33.65
--tserver_master_addrs=127.9.33.126:37671
--webserver_port=42967
--webserver_interface=127.9.33.65
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:35845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:46:07.005913 25295 log.cc:824] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b: Log is configured to *not* fsync() on all Append() calls
I20260928 14:46:07.008574 25295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 14:46:07.008944 25295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b: Bootstrap complete.
I20260928 14:46:07.011816 25295 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5959194892144cb0874ef1b45f8a628b" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37671 } }
I20260928 14:46:07.012297 25295 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5959194892144cb0874ef1b45f8a628b, State: Initialized, Role: FOLLOWER
I20260928 14:46:07.012552 25295 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5959194892144cb0874ef1b45f8a628b" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37671 } }
I20260928 14:46:07.012658 25295 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:46:07.012725 25295 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:46:07.012782 25295 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [term 1 FOLLOWER]: Advancing to term 2
I20260928 14:46:07.014159 25295 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5959194892144cb0874ef1b45f8a628b" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37671 } }
I20260928 14:46:07.014328 25295 leader_election.cc:302] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5959194892144cb0874ef1b45f8a628b; no voters:
I20260928 14:46:07.014565 25295 leader_election.cc:288] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260928 14:46:07.014644 25299 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [term 2 FOLLOWER]: Leader election won for term 2
I20260928 14:46:07.014829 25299 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [term 2 LEADER]: Becoming Leader. State: Replica: 5959194892144cb0874ef1b45f8a628b, State: Running, Role: LEADER
I20260928 14:46:07.014916 25295 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:46:07.014945 25299 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5959194892144cb0874ef1b45f8a628b" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37671 } }
I20260928 14:46:07.015259 25301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5959194892144cb0874ef1b45f8a628b. Latest consensus state: current_term: 2 leader_uuid: "5959194892144cb0874ef1b45f8a628b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5959194892144cb0874ef1b45f8a628b" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37671 } } }
I20260928 14:46:07.015275 25300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5959194892144cb0874ef1b45f8a628b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5959194892144cb0874ef1b45f8a628b" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37671 } } }
I20260928 14:46:07.015462 25300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [sys.catalog]: This master's current role is: LEADER
I20260928 14:46:07.015758 25301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [sys.catalog]: This master's current role is: LEADER
I20260928 14:46:07.016009 25305 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:46:07.017002 25305 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:46:07.017118 25305 catalog_manager.cc:1259] Loaded cluster ID: 1ec87a4d7c7e40a09ac7aa58277427c9
I20260928 14:46:07.017153 25305 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:46:07.018488 25305 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:46:07.018698 25305 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b: Loaded TSK: 0
I20260928 14:46:07.029563 25305 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b: Generated new TSK 1
I20260928 14:46:07.029739 25305 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 14:46:07.105273 25297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:46:07.105629 25297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:46:07.105684 25297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:46:07.107290 25297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:46:07.107398 25297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:46:07.107452 25297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.65
I20260928 14:46:07.109460 25297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:35845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.65:41211
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.33.65
--webserver_port=42967
--tserver_master_addrs=127.9.33.126:37671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.65
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:46:07.109884 25297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:46:07.110219 25297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:46:07.117851 25325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:07.117839 25322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:07.117841 25323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:07.118188 25297 server_base.cc:1034] running on GCE node
I20260928 14:46:07.118614 25297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:46:07.118986 25297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:46:07.120159 25297 hybrid_clock.cc:648] HybridClock initialized: now 1790606767120141 us; error 37 us; skew 500 ppm
Sep 28 14:46:07 dist-test-slave-pv0w krb5kdc[24679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606767, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:46:07.123456 25297 init.cc:377] Logged in from keytab as kudu/127.9.33.65@KRBTEST.COM (short username kudu)
I20260928 14:46:07.124189 25297 webserver.cc:460] Webserver started at http://127.9.33.65:42967/ using document root <none> and password file <none>
I20260928 14:46:07.124513 25297 fs_manager.cc:359] Metadata directory not provided
I20260928 14:46:07.124589 25297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:46:07.126492 25297 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:46:07.127378 25332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:07.127560 25297 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:46:07.127676 25297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/wal
uuid: "eafe320ad6134d8397a349b3e0f30e54"
format_stamp: "Formatted at 2026-09-28 14:45:46 on dist-test-slave-pv0w"
server_key: "54693ff559a2d17a617d166c70065be0"
server_key_iv: "1f946ada8ebc254bf91cd9d9eaa6fa84"
server_key_version: "encryptionkey@0"
I20260928 14:46:07.128202 25297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:46:07.151083 25297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:46:07.154063 25297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:46:07.154299 25297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:46:07.154714 25297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:46:07.155401 25339 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260928 14:46:07.157392 25297 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260928 14:46:07.157459 25297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260928 14:46:07.157532 25297 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260928 14:46:07.158454 25297 ts_tablet_manager.cc:610] Registered 1 tablets
I20260928 14:46:07.158501 25297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260928 14:46:07.158572 25339 tablet_bootstrap.cc:492] T 5e91bed5a5024fe6a693b14a85eec62f P eafe320ad6134d8397a349b3e0f30e54: Bootstrap starting.
I20260928 14:46:07.169402 25297 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.65:41211
I20260928 14:46:07.169533 25446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.65:41211 every 8 connection(s)
I20260928 14:46:07.170199 25297 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
I20260928 14:46:07.173990 25339 log.cc:824] T 5e91bed5a5024fe6a693b14a85eec62f P eafe320ad6134d8397a349b3e0f30e54: Log is configured to *not* fsync() on all Append() calls
I20260928 14:46:07.176211 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 25297
I20260928 14:46:07.176731 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.66:46101
--local_ip_for_outbound_sockets=127.9.33.66
--tserver_master_addrs=127.9.33.126:37671
--webserver_port=35687
--webserver_interface=127.9.33.66
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:35845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:46:07.177883 25339 tablet_bootstrap.cc:492] T 5e91bed5a5024fe6a693b14a85eec62f P eafe320ad6134d8397a349b3e0f30e54: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 14:46:07.178248 25339 tablet_bootstrap.cc:492] T 5e91bed5a5024fe6a693b14a85eec62f P eafe320ad6134d8397a349b3e0f30e54: Bootstrap complete.
I20260928 14:46:07.179596 25339 ts_tablet_manager.cc:1397] T 5e91bed5a5024fe6a693b14a85eec62f P eafe320ad6134d8397a349b3e0f30e54: Time spent bootstrapping tablet: real 0.021s user 0.005s sys 0.012s
I20260928 14:46:07.181138 25339 raft_consensus.cc:348] T 5e91bed5a5024fe6a693b14a85eec62f P eafe320ad6134d8397a349b3e0f30e54 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eafe320ad6134d8397a349b3e0f30e54" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41211 } } peers { permanent_uuid: "41521aceb5454ef4977c9ffd7b683b6b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 46101 } } peers { permanent_uuid: "b86701327a744e368607ded503580579" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34371 } }
I20260928 14:46:07.181454 25339 raft_consensus.cc:729] T 5e91bed5a5024fe6a693b14a85eec62f P eafe320ad6134d8397a349b3e0f30e54 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: eafe320ad6134d8397a349b3e0f30e54, State: Initialized, Role: FOLLOWER
I20260928 14:46:07.181633 25339 consensus_queue.cc:260] T 5e91bed5a5024fe6a693b14a85eec62f P eafe320ad6134d8397a349b3e0f30e54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eafe320ad6134d8397a349b3e0f30e54" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41211 } } peers { permanent_uuid: "41521aceb5454ef4977c9ffd7b683b6b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 46101 } } peers { permanent_uuid: "b86701327a744e368607ded503580579" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34371 } }
I20260928 14:46:07.182017 25339 ts_tablet_manager.cc:1428] T 5e91bed5a5024fe6a693b14a85eec62f P eafe320ad6134d8397a349b3e0f30e54: Time spent starting tablet: real 0.002s user 0.002s sys 0.004s
Sep 28 14:46:07 dist-test-slave-pv0w krb5kdc[24679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606767, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:46:07.187964 25450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:07.171345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:35849 (local address 127.9.33.126:37671)
0928 14:46:07.174366 (+ 3021us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:07.174379 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:07.174427 (+ 48us) server_negotiation.cc:407] Connection header received
0928 14:46:07.174593 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:07.174601 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:07.174661 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:07.174813 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:07.175873 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.177359 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.178866 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.179592 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.179786 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:07.183143 (+ 3357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:07.183149 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:07.183149 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:46:07.183165 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:07.185305 (+ 2140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:07.186020 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:07.186023 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:07.186024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:07.186164 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:07.186559 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:07.186560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:07.186561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:07.186934 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:07.187015 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:07.187518 (+ 503us) server_negotiation.cc:299] Negotiation successful
0928 14:46:07.187702 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2934,"thread_start_us":132,"threads_started":1}
I20260928 14:46:07.188042 25449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:07.171180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.65:35849)
0928 14:46:07.171423 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:07.171479 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:07.172168 (+ 689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:07.174886 (+ 2718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:07.174891 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:07.175359 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:46:07.175689 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:07.175700 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.177510 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.177512 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:07.178741 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:07.178747 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.179774 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.179776 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:07.179975 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:07.180732 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:46:07.180745 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:46:07.182957 (+ 2212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:46:07.185415 (+ 2458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:46:07.185428 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:46:07.185431 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:46:07.185862 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:46:07.186248 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:46:07.186250 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:46:07.186251 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:46:07.186438 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:46:07.187023 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:46:07.187031 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:46:07.187252 (+ 221us) client_negotiation.cc:769] Sending connection context
0928 14:46:07.187392 (+ 140us) client_negotiation.cc:240] Negotiation successful
0928 14:46:07.187662 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":141,"threads_started":1}
I20260928 14:46:07.188995 25447 heartbeater.cc:347] Connected to a master server at 127.9.33.126:37671
I20260928 14:46:07.189157 25447 heartbeater.cc:464] Registering TS with master...
I20260928 14:46:07.189401 25447 heartbeater.cc:511] Master 127.9.33.126:37671 requested a full tablet report, sending...
I20260928 14:46:07.190367 25259 ts_manager.cc:194] Registered new tserver with Master: eafe320ad6134d8397a349b3e0f30e54 (127.9.33.65:41211)
I20260928 14:46:07.191838 25259 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.65@KRBTEST.COM'} at 127.9.33.65:35849
I20260928 14:46:07.193030 25447 heartbeater.cc:503] Master 127.9.33.126:37671 was elected leader, sending a full tablet report...
Sep 28 14:46:07 dist-test-slave-pv0w krb5kdc[24679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606766, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for kudu/127.9.33.65@KRBTEST.COM
I20260928 14:46:07.202113 25454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:07.191230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:41211 (local address 127.0.0.1:56200)
0928 14:46:07.191537 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:07.191580 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:07.191704 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:07.192065 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:07.192072 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:07.192534 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:46:07.192617 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:07.192620 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.194131 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.194133 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:07.194825 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:07.194828 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.195580 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.195581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:07.195707 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:07.196408 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:46:07.196419 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:46:07.198265 (+ 1846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:46:07.200630 (+ 2365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:46:07.200639 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:46:07.200642 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:46:07.200885 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:46:07.201129 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:46:07.201129 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:46:07.201129 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:46:07.201207 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:46:07.201620 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:46:07.201629 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:46:07.201699 (+ 70us) client_negotiation.cc:769] Sending connection context
0928 14:46:07.201798 (+ 99us) client_negotiation.cc:240] Negotiation successful
0928 14:46:07.201956 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":120,"threads_started":1}
I20260928 14:46:07.202126 25455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:07.191324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56200 (local address 127.9.33.65:41211)
0928 14:46:07.191544 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:07.191550 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:07.191668 (+ 118us) server_negotiation.cc:407] Connection header received
0928 14:46:07.191860 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:07.191863 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:07.191881 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:07.191999 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:07.192758 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.194014 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.194912 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.195425 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.195507 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:07.198372 (+ 2865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:07.198387 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:07.198387 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:46:07.198398 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:07.200528 (+ 2130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:07.200978 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:07.200983 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:07.200984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:07.201060 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:07.201270 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:07.201271 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:07.201272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:07.201525 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:07.201617 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:07.201874 (+ 257us) server_negotiation.cc:299] Negotiation successful
0928 14:46:07.202004 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":100,"threads_started":1}
I20260928 14:46:07.202849 25381 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e91bed5a5024fe6a693b14a85eec62f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:45:46 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:56200
I20260928 14:46:07.203526 25456 tablet_replica.cc:330] stopping tablet replica
I20260928 14:46:07.203675 25456 raft_consensus.cc:2229] T 5e91bed5a5024fe6a693b14a85eec62f P eafe320ad6134d8397a349b3e0f30e54 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:46:07.203804 25456 raft_consensus.cc:2258] T 5e91bed5a5024fe6a693b14a85eec62f P eafe320ad6134d8397a349b3e0f30e54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:46:07.204245 25456 ts_tablet_manager.cc:1905] T 5e91bed5a5024fe6a693b14a85eec62f P eafe320ad6134d8397a349b3e0f30e54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:46:07.206897 25456 ts_tablet_manager.cc:1918] T 5e91bed5a5024fe6a693b14a85eec62f P eafe320ad6134d8397a349b3e0f30e54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:46:07.207001 25456 log.cc:1196] T 5e91bed5a5024fe6a693b14a85eec62f P eafe320ad6134d8397a349b3e0f30e54: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/wal/wals/5e91bed5a5024fe6a693b14a85eec62f
I20260928 14:46:07.207257 25456 ts_tablet_manager.cc:1939] T 5e91bed5a5024fe6a693b14a85eec62f P eafe320ad6134d8397a349b3e0f30e54: Deleting consensus metadata
I20260928 14:46:07.207932 25246 catalog_manager.cc:4841] TS eafe320ad6134d8397a349b3e0f30e54 (127.9.33.65:41211): tablet 5e91bed5a5024fe6a693b14a85eec62f (table test-table [id=9bc042ced5c048de9f70411bee32d7cb]) successfully deleted
W20260928 14:46:07.280853 25452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:46:07.281168 25452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:46:07.281199 25452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:46:07.282760 25452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:46:07.282829 25452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:46:07.282866 25452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.66
I20260928 14:46:07.284849 25452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:35845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.66:46101
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.33.66
--webserver_port=35687
--tserver_master_addrs=127.9.33.126:37671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.66
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:46:07.285228 25452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:46:07.285558 25452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:46:07.292058 25464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:07.292050 25461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:07.292050 25462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:07.292554 25452 server_base.cc:1034] running on GCE node
I20260928 14:46:07.292780 25452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:46:07.293083 25452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:46:07.294274 25452 hybrid_clock.cc:648] HybridClock initialized: now 1790606767294259 us; error 33 us; skew 500 ppm
Sep 28 14:46:07 dist-test-slave-pv0w krb5kdc[24679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606767, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:46:07.297571 25452 init.cc:377] Logged in from keytab as kudu/127.9.33.66@KRBTEST.COM (short username kudu)
I20260928 14:46:07.298174 25452 webserver.cc:460] Webserver started at http://127.9.33.66:35687/ using document root <none> and password file <none>
I20260928 14:46:07.298434 25452 fs_manager.cc:359] Metadata directory not provided
I20260928 14:46:07.298499 25452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:46:07.300377 25452 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:46:07.301203 25471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:07.301590 25452 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:46:07.301700 25452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/wal
uuid: "41521aceb5454ef4977c9ffd7b683b6b"
format_stamp: "Formatted at 2026-09-28 14:45:46 on dist-test-slave-pv0w"
server_key: "c0465e976428b8a74d6866d2b5916d33"
server_key_iv: "cdaf2e85d120be8e711704a8a720e34d"
server_key_version: "encryptionkey@0"
I20260928 14:46:07.302176 25452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:46:07.314451 25452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:46:07.317902 25452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:46:07.318097 25452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:46:07.318418 25452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:46:07.318815 25452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:46:07.318852 25452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:07.318903 25452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:46:07.318929 25452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:07.326874 25452 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.66:46101
I20260928 14:46:07.326992 25584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.66:46101 every 8 connection(s)
I20260928 14:46:07.327445 25452 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
I20260928 14:46:07.334950 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 25452
I20260928 14:46:07.335407 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.67:34371
--local_ip_for_outbound_sockets=127.9.33.67
--tserver_master_addrs=127.9.33.126:37671
--webserver_port=36613
--webserver_interface=127.9.33.67
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:35845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 28 14:46:07 dist-test-slave-pv0w krb5kdc[24679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606767, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:46:07.342483 25450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:07.328595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:39089 (local address 127.9.33.126:37671)
0928 14:46:07.328697 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:07.328700 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:07.328907 (+ 207us) server_negotiation.cc:407] Connection header received
0928 14:46:07.329685 (+ 778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:07.329687 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:07.329722 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:07.329803 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:07.331289 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.332164 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.333663 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.333933 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.334050 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:07.338062 (+ 4012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:07.338066 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:07.338066 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:46:07.338078 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:07.340079 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:07.340836 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:07.340838 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:07.340839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:07.340904 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:07.341306 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:07.341307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:07.341308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:07.341586 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:07.341666 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:07.342220 (+ 554us) server_negotiation.cc:299] Negotiation successful
0928 14:46:07.342388 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260928 14:46:07.342833 25587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:07.328474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.66:39089)
0928 14:46:07.328704 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:07.328770 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:07.329497 (+ 727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:07.329910 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:07.329924 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:07.330748 (+ 824us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:46:07.331115 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:07.331126 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.332378 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.332380 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:07.333522 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:07.333526 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.334076 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.334078 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:07.334274 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:07.335403 (+ 1129us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:46:07.335423 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:46:07.337866 (+ 2443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:46:07.340223 (+ 2357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:46:07.340238 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:46:07.340242 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:46:07.340717 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:46:07.340992 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:46:07.340994 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:46:07.340994 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:46:07.341194 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:46:07.341707 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:46:07.341718 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:46:07.342013 (+ 295us) client_negotiation.cc:769] Sending connection context
0928 14:46:07.342182 (+ 169us) client_negotiation.cc:240] Negotiation successful
0928 14:46:07.342439 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":76,"threads_started":1}
I20260928 14:46:07.343617 25585 heartbeater.cc:347] Connected to a master server at 127.9.33.126:37671
I20260928 14:46:07.343830 25585 heartbeater.cc:464] Registering TS with master...
I20260928 14:46:07.344122 25585 heartbeater.cc:511] Master 127.9.33.126:37671 requested a full tablet report, sending...
I20260928 14:46:07.344714 25259 ts_manager.cc:194] Registered new tserver with Master: 41521aceb5454ef4977c9ffd7b683b6b (127.9.33.66:46101)
I20260928 14:46:07.345324 25259 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.66@KRBTEST.COM'} at 127.9.33.66:39089
W20260928 14:46:07.428143 25588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:46:07.428380 25588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:46:07.428406 25588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:46:07.429540 25588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:46:07.429584 25588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:46:07.429610 25588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.67
I20260928 14:46:07.431054 25588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:35845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.67:34371
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.33.67
--webserver_port=36613
--tserver_master_addrs=127.9.33.126:37671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.67
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:46:07.431260 25588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:46:07.431550 25588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:46:07.437371 25594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:07.437300 25593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:07.437325 25588 server_base.cc:1034] running on GCE node
W20260928 14:46:07.437304 25596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:07.437810 25588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:46:07.438112 25588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:46:07.439297 25588 hybrid_clock.cc:648] HybridClock initialized: now 1790606767439274 us; error 44 us; skew 500 ppm
Sep 28 14:46:07 dist-test-slave-pv0w krb5kdc[24679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606767, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:46:07.442057 25588 init.cc:377] Logged in from keytab as kudu/127.9.33.67@KRBTEST.COM (short username kudu)
I20260928 14:46:07.442629 25588 webserver.cc:460] Webserver started at http://127.9.33.67:36613/ using document root <none> and password file <none>
I20260928 14:46:07.442883 25588 fs_manager.cc:359] Metadata directory not provided
I20260928 14:46:07.442972 25588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:46:07.444566 25588 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260928 14:46:07.445199 25603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:07.445544 25588 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:46:07.445667 25588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/wal
uuid: "b86701327a744e368607ded503580579"
format_stamp: "Formatted at 2026-09-28 14:45:46 on dist-test-slave-pv0w"
server_key: "1745a03d9e9a59c21a864b52d9a5d2bd"
server_key_iv: "6296c5ef37a097596ab410c48d3b9a90"
server_key_version: "encryptionkey@0"
I20260928 14:46:07.446044 25588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:46:07.461114 25588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:46:07.463472 25588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:46:07.463650 25588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:46:07.463948 25588 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:46:07.464298 25588 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:46:07.464385 25588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:07.464432 25588 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:46:07.464458 25588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:07.470369 25588 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.67:34371
I20260928 14:46:07.470438 25716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.67:34371 every 8 connection(s)
I20260928 14:46:07.470839 25588 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
I20260928 14:46:07.472496 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 25588
Sep 28 14:46:07 dist-test-slave-pv0w krb5kdc[24679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606767, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:46:07.480011 25721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:07.472944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33254 (local address 127.9.33.126:37671)
0928 14:46:07.473081 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:07.473084 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:07.473117 (+ 33us) server_negotiation.cc:407] Connection header received
0928 14:46:07.473203 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:07.473203 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:07.473215 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:07.473311 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:07.473948 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.474447 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.475215 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.475458 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.475540 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:07.476935 (+ 1395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:07.476937 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:07.476937 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:46:07.476944 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:07.478589 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:07.479012 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:07.479013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:07.479014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:07.479097 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:07.479317 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:07.479318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:07.479318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:07.479482 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:07.479618 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:07.479786 (+ 168us) server_negotiation.cc:299] Negotiation successful
0928 14:46:07.479890 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":80,"threads_started":1}
I20260928 14:46:07.481364 25450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:07.471694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:53933 (local address 127.9.33.126:37671)
0928 14:46:07.471797 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:07.471799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:07.472046 (+ 247us) server_negotiation.cc:407] Connection header received
0928 14:46:07.472703 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:07.472705 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:07.472719 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:07.472774 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:07.473876 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.474409 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.475458 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.475662 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.475773 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:07.478054 (+ 2281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:07.478056 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:07.478056 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:46:07.478063 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:07.479821 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:07.480353 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:07.480354 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:07.480355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:07.480416 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:07.480708 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:07.480709 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:07.480709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:07.480847 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:07.480911 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:07.481230 (+ 319us) server_negotiation.cc:299] Negotiation successful
0928 14:46:07.481303 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260928 14:46:07.481535 25719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:07.471701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.67:53933)
0928 14:46:07.471919 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:07.471957 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:07.472526 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:07.472860 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:07.472864 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:07.473549 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:46:07.473761 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:07.473766 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.474515 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.474516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:07.475347 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:07.475349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.475794 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.475796 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:07.475928 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:07.476475 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:46:07.476484 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:46:07.477930 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:46:07.479921 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:46:07.479930 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:46:07.479933 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:46:07.480252 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:46:07.480493 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:46:07.480494 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:46:07.480494 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:46:07.480633 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:46:07.480915 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:46:07.480920 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:46:07.481101 (+ 181us) client_negotiation.cc:769] Sending connection context
0928 14:46:07.481190 (+ 89us) client_negotiation.cc:240] Negotiation successful
0928 14:46:07.481325 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":82,"threads_started":1}
I20260928 14:46:07.482003 25717 heartbeater.cc:347] Connected to a master server at 127.9.33.126:37671
I20260928 14:46:07.482111 25717 heartbeater.cc:464] Registering TS with master...
I20260928 14:46:07.482301 25717 heartbeater.cc:511] Master 127.9.33.126:37671 requested a full tablet report, sending...
I20260928 14:46:07.482677 25259 ts_manager.cc:194] Registered new tserver with Master: b86701327a744e368607ded503580579 (127.9.33.67:34371)
I20260928 14:46:07.483016 25259 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.67@KRBTEST.COM'} at 127.9.33.67:53933
I20260928 14:46:07.492035 9349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:46:07.494776 25450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:07.492428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33264 (local address 127.9.33.126:37671)
0928 14:46:07.492474 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:07.492476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:07.492574 (+ 98us) server_negotiation.cc:407] Connection header received
0928 14:46:07.492651 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:07.492652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:07.492690 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:07.492758 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:46:07.492869 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.493350 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.494135 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.494319 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.494375 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:07.494541 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:46:07.494587 (+ 46us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0928 14:46:07.494716 (+ 129us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33264: token expired
Metrics: {"server-negotiator.queue_time_us":21}
W20260928 14:46:07.494874 25450 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33264: token expired
W20260928 14:46:07.495036 25187 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.9.33.126:37671)
I20260928 14:46:07.501503 25450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:07.495264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33276 (local address 127.9.33.126:37671)
0928 14:46:07.495310 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:07.495312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:07.495321 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:46:07.495365 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:07.495366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:07.495376 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:07.495469 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:07.496099 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.496649 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.497326 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.497519 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.497575 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:07.498750 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:07.498751 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:07.498751 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:46:07.498758 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:07.500442 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:07.500826 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:07.500827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:07.500828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:07.500883 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:07.501096 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:07.501097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:07.501097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:07.501200 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:07.501280 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:07.501371 (+ 91us) server_negotiation.cc:299] Negotiation successful
0928 14:46:07.501434 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260928 14:46:07.502769 25259 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33276:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:46:07.504698 25259 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:46:07.507012 25381 tablet_service.cc:1467] Processing CreateTablet for tablet 57f4cbaad13c4520a364de9f24910b1f (DEFAULT_TABLE table=test-table [id=3f4f89ca1d9e49a980e1128af05ffc48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:46:07.507284 25381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57f4cbaad13c4520a364de9f24910b1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:46:07.509068 25339 tablet_bootstrap.cc:492] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54: Bootstrap starting.
I20260928 14:46:07.509394 25339 tablet_bootstrap.cc:654] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54: Neither blocks nor log segments found. Creating new log.
I20260928 14:46:07.510020 25339 tablet_bootstrap.cc:492] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54: No bootstrap required, opened a new log
I20260928 14:46:07.510071 25339 ts_tablet_manager.cc:1397] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260928 14:46:07.510450 25339 raft_consensus.cc:348] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86701327a744e368607ded503580579" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34371 } } peers { permanent_uuid: "41521aceb5454ef4977c9ffd7b683b6b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 46101 } } peers { permanent_uuid: "eafe320ad6134d8397a349b3e0f30e54" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41211 } }
I20260928 14:46:07.510535 25339 raft_consensus.cc:374] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:46:07.510581 25339 raft_consensus.cc:729] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eafe320ad6134d8397a349b3e0f30e54, State: Initialized, Role: FOLLOWER
I20260928 14:46:07.510632 25339 consensus_queue.cc:260] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86701327a744e368607ded503580579" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34371 } } peers { permanent_uuid: "41521aceb5454ef4977c9ffd7b683b6b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 46101 } } peers { permanent_uuid: "eafe320ad6134d8397a349b3e0f30e54" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41211 } }
I20260928 14:46:07.510725 25339 ts_tablet_manager.cc:1428] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260928 14:46:07.510772 25723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:07.506692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:34371 (local address 127.0.0.1:58084)
0928 14:46:07.506858 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:07.506872 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:07.506936 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:07.507335 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:07.507336 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:07.507337 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:46:07.507408 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:07.507410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.508396 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.508397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:07.509421 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:07.509424 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.510467 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.510468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:07.510534 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:07.510558 (+ 24us) client_negotiation.cc:769] Sending connection context
0928 14:46:07.510623 (+ 65us) client_negotiation.cc:240] Negotiation successful
0928 14:46:07.510669 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126}
I20260928 14:46:07.510787 25724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:07.506788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58084 (local address 127.9.33.67:34371)
0928 14:46:07.507174 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:07.507180 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:07.507191 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:46:07.507232 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:07.507234 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:07.507252 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:07.507329 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:46:07.507472 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.508297 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.509526 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.510373 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.510452 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:07.510515 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:07.510650 (+ 135us) server_negotiation.cc:299] Negotiation successful
0928 14:46:07.510697 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":86,"threads_started":1}
I20260928 14:46:07.511023 25454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:07.506709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:46101 (local address 127.0.0.1:45236)
0928 14:46:07.506981 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:07.506992 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:07.507039 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:07.507507 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:07.507508 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:07.507510 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:46:07.507578 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:07.507580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.508901 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.508902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:07.509938 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:07.509941 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.510780 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.510781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:07.510857 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:07.510861 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:46:07.510895 (+ 34us) client_negotiation.cc:240] Negotiation successful
0928 14:46:07.510953 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":82,"threads_started":1}
I20260928 14:46:07.511343 25725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:07.506989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45236 (local address 127.9.33.66:46101)
0928 14:46:07.507238 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:07.507243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:07.507255 (+ 12us) server_negotiation.cc:407] Connection header received
0928 14:46:07.507295 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:07.507298 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:07.507314 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:07.507654 (+ 340us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:46:07.507764 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.508788 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.510040 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.510692 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.511113 (+ 421us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:07.511179 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:07.511205 (+ 26us) server_negotiation.cc:299] Negotiation successful
0928 14:46:07.511245 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":94,"threads_started":1}
I20260928 14:46:07.511577 25651 tablet_service.cc:1467] Processing CreateTablet for tablet 57f4cbaad13c4520a364de9f24910b1f (DEFAULT_TABLE table=test-table [id=3f4f89ca1d9e49a980e1128af05ffc48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:46:07.511999 25651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57f4cbaad13c4520a364de9f24910b1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:46:07.512038 25519 tablet_service.cc:1467] Processing CreateTablet for tablet 57f4cbaad13c4520a364de9f24910b1f (DEFAULT_TABLE table=test-table [id=3f4f89ca1d9e49a980e1128af05ffc48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:46:07.512374 25519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57f4cbaad13c4520a364de9f24910b1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:46:07.515877 25728 tablet_bootstrap.cc:492] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579: Bootstrap starting.
I20260928 14:46:07.515877 25727 tablet_bootstrap.cc:492] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b: Bootstrap starting.
I20260928 14:46:07.516839 25728 tablet_bootstrap.cc:654] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579: Neither blocks nor log segments found. Creating new log.
I20260928 14:46:07.516856 25727 tablet_bootstrap.cc:654] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b: Neither blocks nor log segments found. Creating new log.
I20260928 14:46:07.517519 25727 log.cc:824] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b: Log is configured to *not* fsync() on all Append() calls
I20260928 14:46:07.517530 25728 log.cc:824] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579: Log is configured to *not* fsync() on all Append() calls
I20260928 14:46:07.518328 25727 tablet_bootstrap.cc:492] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b: No bootstrap required, opened a new log
I20260928 14:46:07.518407 25727 ts_tablet_manager.cc:1397] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260928 14:46:07.518496 25728 tablet_bootstrap.cc:492] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579: No bootstrap required, opened a new log
I20260928 14:46:07.518558 25728 ts_tablet_manager.cc:1397] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260928 14:46:07.520504 25728 raft_consensus.cc:348] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86701327a744e368607ded503580579" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34371 } } peers { permanent_uuid: "41521aceb5454ef4977c9ffd7b683b6b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 46101 } } peers { permanent_uuid: "eafe320ad6134d8397a349b3e0f30e54" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41211 } }
I20260928 14:46:07.520504 25727 raft_consensus.cc:348] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86701327a744e368607ded503580579" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34371 } } peers { permanent_uuid: "41521aceb5454ef4977c9ffd7b683b6b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 46101 } } peers { permanent_uuid: "eafe320ad6134d8397a349b3e0f30e54" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41211 } }
I20260928 14:46:07.520709 25727 raft_consensus.cc:374] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:46:07.520709 25728 raft_consensus.cc:374] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:46:07.520748 25728 raft_consensus.cc:729] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b86701327a744e368607ded503580579, State: Initialized, Role: FOLLOWER
I20260928 14:46:07.520748 25727 raft_consensus.cc:729] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41521aceb5454ef4977c9ffd7b683b6b, State: Initialized, Role: FOLLOWER
I20260928 14:46:07.520875 25728 consensus_queue.cc:260] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86701327a744e368607ded503580579" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34371 } } peers { permanent_uuid: "41521aceb5454ef4977c9ffd7b683b6b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 46101 } } peers { permanent_uuid: "eafe320ad6134d8397a349b3e0f30e54" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41211 } }
I20260928 14:46:07.520875 25727 consensus_queue.cc:260] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86701327a744e368607ded503580579" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34371 } } peers { permanent_uuid: "41521aceb5454ef4977c9ffd7b683b6b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 46101 } } peers { permanent_uuid: "eafe320ad6134d8397a349b3e0f30e54" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41211 } }
I20260928 14:46:07.521190 25717 heartbeater.cc:503] Master 127.9.33.126:37671 was elected leader, sending a full tablet report...
I20260928 14:46:07.521190 25585 heartbeater.cc:503] Master 127.9.33.126:37671 was elected leader, sending a full tablet report...
I20260928 14:46:07.521441 25727 ts_tablet_manager.cc:1428] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260928 14:46:07.521441 25728 ts_tablet_manager.cc:1428] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260928 14:46:07.578174 25586 tablet.cc:2369] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:46:07.585623 25732 raft_consensus.cc:482] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:46:07.585757 25732 raft_consensus.cc:504] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86701327a744e368607ded503580579" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34371 } } peers { permanent_uuid: "41521aceb5454ef4977c9ffd7b683b6b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 46101 } } peers { permanent_uuid: "eafe320ad6134d8397a349b3e0f30e54" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41211 } }
I20260928 14:46:07.586143 25732 leader_election.cc:288] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41521aceb5454ef4977c9ffd7b683b6b (127.9.33.66:46101), eafe320ad6134d8397a349b3e0f30e54 (127.9.33.65:41211)
I20260928 14:46:07.589329 25725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:07.586468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:57653 (local address 127.9.33.66:46101)
0928 14:46:07.586512 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:07.586514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:07.586522 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:46:07.586550 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:07.586551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:07.586563 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:07.586622 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:46:07.586847 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.587402 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.588516 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.588953 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.589010 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:07.589020 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:07.589209 (+ 189us) server_negotiation.cc:299] Negotiation successful
0928 14:46:07.589250 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 14:46:07.589313 25719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:07.586335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:46101 (local address 127.9.33.67:57653)
0928 14:46:07.586401 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:07.586413 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:07.586469 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:07.586649 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:07.586651 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:07.586653 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:46:07.586740 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:07.586743 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.587489 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.587490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:07.588401 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:07.588404 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.589036 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.589037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:07.589120 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:07.589124 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:46:07.589185 (+ 61us) client_negotiation.cc:240] Negotiation successful
0928 14:46:07.589225 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":28}
I20260928 14:46:07.589483 25733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:07.586335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:41211 (local address 127.9.33.67:52001)
0928 14:46:07.586628 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:07.586641 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:07.586684 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:07.586848 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:07.586849 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:07.586850 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:46:07.586925 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:07.586926 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.587682 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.587683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:07.588554 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:07.588556 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.589218 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.589219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:07.589287 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:07.589291 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:46:07.589352 (+ 61us) client_negotiation.cc:240] Negotiation successful
0928 14:46:07.589394 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":84,"thread_start_us":146,"threads_started":1}
I20260928 14:46:07.589495 25455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:07.586444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:52001 (local address 127.9.33.65:41211)
0928 14:46:07.586508 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:07.586511 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:07.586703 (+ 192us) server_negotiation.cc:407] Connection header received
0928 14:46:07.586736 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:07.586737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:07.586767 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:07.586835 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:46:07.587018 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.587601 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.588642 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.589119 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.589194 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:07.589277 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:07.589373 (+ 96us) server_negotiation.cc:299] Negotiation successful
0928 14:46:07.589416 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260928 14:46:07.589828 25539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57f4cbaad13c4520a364de9f24910b1f" candidate_uuid: "b86701327a744e368607ded503580579" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41521aceb5454ef4977c9ffd7b683b6b" is_pre_election: true
I20260928 14:46:07.589874 25401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57f4cbaad13c4520a364de9f24910b1f" candidate_uuid: "b86701327a744e368607ded503580579" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eafe320ad6134d8397a349b3e0f30e54" is_pre_election: true
I20260928 14:46:07.590005 25401 raft_consensus.cc:2454] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b86701327a744e368607ded503580579 in term 0.
I20260928 14:46:07.590006 25539 raft_consensus.cc:2454] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b86701327a744e368607ded503580579 in term 0.
I20260928 14:46:07.590262 25605 leader_election.cc:302] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41521aceb5454ef4977c9ffd7b683b6b, b86701327a744e368607ded503580579; no voters:
I20260928 14:46:07.590451 25732 raft_consensus.cc:2781] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:46:07.590493 25732 raft_consensus.cc:482] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:46:07.590509 25732 raft_consensus.cc:3037] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:46:07.591215 25732 raft_consensus.cc:504] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86701327a744e368607ded503580579" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34371 } } peers { permanent_uuid: "41521aceb5454ef4977c9ffd7b683b6b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 46101 } } peers { permanent_uuid: "eafe320ad6134d8397a349b3e0f30e54" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41211 } }
I20260928 14:46:07.591387 25732 leader_election.cc:288] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579 [CANDIDATE]: Term 1 election: Requested vote from peers 41521aceb5454ef4977c9ffd7b683b6b (127.9.33.66:46101), eafe320ad6134d8397a349b3e0f30e54 (127.9.33.65:41211)
I20260928 14:46:07.591576 25539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57f4cbaad13c4520a364de9f24910b1f" candidate_uuid: "b86701327a744e368607ded503580579" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41521aceb5454ef4977c9ffd7b683b6b"
I20260928 14:46:07.591578 25401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57f4cbaad13c4520a364de9f24910b1f" candidate_uuid: "b86701327a744e368607ded503580579" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eafe320ad6134d8397a349b3e0f30e54"
I20260928 14:46:07.591658 25539 raft_consensus.cc:3037] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:46:07.591658 25401 raft_consensus.cc:3037] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:46:07.592574 25539 raft_consensus.cc:2454] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b86701327a744e368607ded503580579 in term 1.
I20260928 14:46:07.592584 25401 raft_consensus.cc:2454] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b86701327a744e368607ded503580579 in term 1.
I20260928 14:46:07.592751 25607 leader_election.cc:302] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b86701327a744e368607ded503580579, eafe320ad6134d8397a349b3e0f30e54; no voters:
I20260928 14:46:07.592891 25732 raft_consensus.cc:2781] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:46:07.593039 25732 raft_consensus.cc:686] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579 [term 1 LEADER]: Becoming Leader. State: Replica: b86701327a744e368607ded503580579, State: Running, Role: LEADER
I20260928 14:46:07.593146 25732 consensus_queue.cc:237] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86701327a744e368607ded503580579" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34371 } } peers { permanent_uuid: "41521aceb5454ef4977c9ffd7b683b6b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 46101 } } peers { permanent_uuid: "eafe320ad6134d8397a349b3e0f30e54" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41211 } }
I20260928 14:46:07.593971 25258 catalog_manager.cc:5495] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579 reported cstate change: term changed from 0 to 1, leader changed from <none> to b86701327a744e368607ded503580579 (127.9.33.67). New cstate: current_term: 1 leader_uuid: "b86701327a744e368607ded503580579" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86701327a744e368607ded503580579" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34371 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41521aceb5454ef4977c9ffd7b683b6b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 46101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eafe320ad6134d8397a349b3e0f30e54" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41211 } health_report { overall_health: UNKNOWN } } }
I20260928 14:46:07.609342 25724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:07.606822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58098 (local address 127.9.33.67:34371)
0928 14:46:07.606908 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:07.606910 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:07.606920 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:46:07.606961 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:07.606962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:07.606984 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:07.607070 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:46:07.607211 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.607696 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.608490 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:07.608707 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:07.608794 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:07.608968 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:46:07.609085 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:46:07.609185 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:07.609204 (+ 19us) server_negotiation.cc:299] Negotiation successful
0928 14:46:07.609226 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260928 14:46:07.611263 25539 raft_consensus.cc:1261] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b [term 1 FOLLOWER]: Refusing update from remote peer b86701327a744e368607ded503580579: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:46:07.611301 25401 raft_consensus.cc:1261] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [term 1 FOLLOWER]: Refusing update from remote peer b86701327a744e368607ded503580579: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:46:07.611654 25732 consensus_queue.cc:1035] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41521aceb5454ef4977c9ffd7b683b6b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 46101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:46:07.611830 25734 consensus_queue.cc:1035] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eafe320ad6134d8397a349b3e0f30e54" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:46:07.614207 25736 mvcc.cc:205] Tried to move back new op lower bound from 7334325320132997120 to 7334325320062259200. Current Snapshot: MvccSnapshot[applied={T|T < 7334325320132997120}]
I20260928 14:46:07.615018 25737 mvcc.cc:205] Tried to move back new op lower bound from 7334325320132997120 to 7334325320062259200. Current Snapshot: MvccSnapshot[applied={T|T < 7334325320132997120}]
I20260928 14:46:07.615216 25738 mvcc.cc:205] Tried to move back new op lower bound from 7334325320132997120 to 7334325320062259200. Current Snapshot: MvccSnapshot[applied={T|T < 7334325320132997120 or (T in {7334325320132997120})}]
I20260928 14:46:07.617236 25258 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:46:07.617350 25258 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:46:07.618136 25258 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b: Sending DeleteTablet for 3 replicas of tablet 57f4cbaad13c4520a364de9f24910b1f
I20260928 14:46:07.618439 25651 tablet_service.cc:1514] Processing DeleteTablet for tablet 57f4cbaad13c4520a364de9f24910b1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:46:07 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:58084
I20260928 14:46:07.618608 25519 tablet_service.cc:1514] Processing DeleteTablet for tablet 57f4cbaad13c4520a364de9f24910b1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:46:07 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:45236
I20260928 14:46:07.618645 25381 tablet_service.cc:1514] Processing DeleteTablet for tablet 57f4cbaad13c4520a364de9f24910b1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:46:07 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:56200
I20260928 14:46:07.618737 25456 tablet_replica.cc:330] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54: stopping tablet replica
I20260928 14:46:07.618742 25745 tablet_replica.cc:330] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579: stopping tablet replica
I20260928 14:46:07.618788 25456 raft_consensus.cc:2229] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:46:07.618840 25456 raft_consensus.cc:2258] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:46:07.618830 25745 raft_consensus.cc:2229] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:46:07.618965 25745 raft_consensus.cc:2258] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:46:07.619058 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 25297
I20260928 14:46:07.619410 25745 ts_tablet_manager.cc:1905] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:46:07.621551 25745 ts_tablet_manager.cc:1918] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:46:07.621639 25745 log.cc:1196] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/wal/wals/57f4cbaad13c4520a364de9f24910b1f
I20260928 14:46:07.621945 25745 ts_tablet_manager.cc:1939] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579: Deleting consensus metadata
I20260928 14:46:07.622920 25243 catalog_manager.cc:4841] TS b86701327a744e368607ded503580579 (127.9.33.67:34371): tablet 57f4cbaad13c4520a364de9f24910b1f (table test-table [id=3f4f89ca1d9e49a980e1128af05ffc48]) successfully deleted
W20260928 14:46:07.626111 25246 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.33.65:41211 (error 108)
I20260928 14:46:07.626333 25746 tablet_replica.cc:330] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b: stopping tablet replica
I20260928 14:46:07.626555 25746 raft_consensus.cc:2229] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:46:07.626547 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 25452
I20260928 14:46:07.626674 25454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:07.626533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:41211 (local address 127.0.0.1:56210)
0928 14:46:07.626594 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:07.626631 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.65:41211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260928 14:46:07.627626 25246 catalog_manager.cc:4584] TS eafe320ad6134d8397a349b3e0f30e54 (127.9.33.65:41211): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 57f4cbaad13c4520a364de9f24910b1f: Network error: Client connection negotiation failed: client connection to 127.9.33.65:41211: connect: Connection refused (error 111)
I20260928 14:46:07.631320 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 25588
I20260928 14:46:07.631349 25454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:07.631206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:46101 (local address 127.0.0.1:45238)
0928 14:46:07.631268 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:07.631308 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.66:46101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260928 14:46:07.636710 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 25226
I20260928 14:46:27.644013 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:37671
--webserver_interface=127.9.33.126
--webserver_port=38113
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:35845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:37671
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:46:27.742574 25748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:46:27.742906 25748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:46:27.742950 25748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:46:27.744287 25748 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260928 14:46:27.744360 25748 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260928 14:46:27.744385 25748 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260928 14:46:27.744405 25748 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:46:27.744428 25748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:46:27.744455 25748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:46:27.744488 25748 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:46:27.744515 25748 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:46:27.746258 25748 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:35845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:37671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:37671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=38113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:46:27.746590 25748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:46:27.746877 25748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:46:27.752847 25753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:27.752839 25754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:27.752839 25756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:27.753196 25748 server_base.cc:1034] running on GCE node
I20260928 14:46:27.753552 25748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:46:27.753966 25748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:46:27.755126 25748 hybrid_clock.cc:648] HybridClock initialized: now 1790606787755111 us; error 53 us; skew 500 ppm
Sep 28 14:46:27 dist-test-slave-pv0w krb5kdc[24679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606787, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:46:27.758371 25748 init.cc:377] Logged in from keytab as kudu/127.9.33.126@KRBTEST.COM (short username kudu)
I20260928 14:46:27.758937 25748 webserver.cc:460] Webserver started at http://127.9.33.126:38113/ using document root <none> and password file <none>
I20260928 14:46:27.759297 25748 fs_manager.cc:359] Metadata directory not provided
I20260928 14:46:27.759378 25748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:46:27.761094 25748 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:46:27.761878 25763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:27.762084 25748 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:46:27.762205 25748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "5959194892144cb0874ef1b45f8a628b"
format_stamp: "Formatted at 2026-09-28 14:45:46 on dist-test-slave-pv0w"
server_key: "fdea5ab4ddc9f7c2ad8ff985b067980e"
server_key_iv: "284d1a15ef73c20cc2603e8d5e0ad843"
server_key_version: "encryptionkey@0"
I20260928 14:46:27.762686 25748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:46:27.779070 25748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:46:27.782258 25748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:46:27.782570 25748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:46:27.787678 25748 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:37671
I20260928 14:46:27.787778 25815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:37671 every 8 connection(s)
I20260928 14:46:27.788309 25748 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:46:27.790160 25816 sys_catalog.cc:263] Verifying existing consensus state
I20260928 14:46:27.790933 25816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b: Bootstrap starting.
I20260928 14:46:27.791249 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 25748
I20260928 14:46:27.791669 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.65:41211
--local_ip_for_outbound_sockets=127.9.33.65
--tserver_master_addrs=127.9.33.126:37671
--webserver_port=42967
--webserver_interface=127.9.33.65
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:35845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:46:27.803046 25816 log.cc:824] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b: Log is configured to *not* fsync() on all Append() calls
I20260928 14:46:27.806196 25816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 14:46:27.806699 25816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b: Bootstrap complete.
I20260928 14:46:27.809504 25816 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5959194892144cb0874ef1b45f8a628b" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37671 } }
I20260928 14:46:27.809823 25816 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5959194892144cb0874ef1b45f8a628b, State: Initialized, Role: FOLLOWER
I20260928 14:46:27.810046 25816 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5959194892144cb0874ef1b45f8a628b" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37671 } }
I20260928 14:46:27.810138 25816 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:46:27.810209 25816 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:46:27.810268 25816 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [term 2 FOLLOWER]: Advancing to term 3
I20260928 14:46:27.811779 25816 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5959194892144cb0874ef1b45f8a628b" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37671 } }
I20260928 14:46:27.811950 25816 leader_election.cc:302] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5959194892144cb0874ef1b45f8a628b; no voters:
I20260928 14:46:27.812213 25816 leader_election.cc:288] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [CANDIDATE]: Term 3 election: Requested vote from peers
I20260928 14:46:27.812264 25820 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [term 3 FOLLOWER]: Leader election won for term 3
I20260928 14:46:27.812417 25820 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [term 3 LEADER]: Becoming Leader. State: Replica: 5959194892144cb0874ef1b45f8a628b, State: Running, Role: LEADER
I20260928 14:46:27.812523 25820 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5959194892144cb0874ef1b45f8a628b" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37671 } }
I20260928 14:46:27.812582 25816 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:46:27.812893 25820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5959194892144cb0874ef1b45f8a628b. Latest consensus state: current_term: 3 leader_uuid: "5959194892144cb0874ef1b45f8a628b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5959194892144cb0874ef1b45f8a628b" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37671 } } }
I20260928 14:46:27.812981 25820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [sys.catalog]: This master's current role is: LEADER
I20260928 14:46:27.812902 25822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "5959194892144cb0874ef1b45f8a628b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5959194892144cb0874ef1b45f8a628b" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37671 } } }
I20260928 14:46:27.813185 25822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b [sys.catalog]: This master's current role is: LEADER
I20260928 14:46:27.814214 25824 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:46:27.815418 25824 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:46:27.815554 25824 catalog_manager.cc:1259] Loaded cluster ID: 1ec87a4d7c7e40a09ac7aa58277427c9
I20260928 14:46:27.815603 25824 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:46:27.817008 25824 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:46:27.817236 25824 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b: Loaded TSK: 0
I20260928 14:46:27.817297 25824 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5959194892144cb0874ef1b45f8a628b: Loaded TSK: 1
F20260928 14:46:27.817598 25824 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f621dc869ad google::LogMessage::Fail() at ??:0
@ 0x7f621dc8988a google::LogMessage::SendToLog() at ??:0
@ 0x7f621dc864de google::LogMessage::Flush() at ??:0
@ 0x7f621dc87a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f62203f3104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f62203f0d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f6225be01cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f6225be048e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f6225be0594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f6225ba6c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f6225bb0ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f621fb540d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f621fb49954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f62209ad6db start_thread at ??:0
@ 0x7f621c78461f clone at ??:0
W20260928 14:46:27.911484 25818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:46:27.911792 25818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:46:27.911836 25818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:46:27.913746 25818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:46:27.913831 25818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:46:27.913877 25818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.65
I20260928 14:46:27.916318 25818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:35845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.65:41211
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.33.65
--webserver_port=42967
--tserver_master_addrs=127.9.33.126:37671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.65
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:46:27.916623 25818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:46:27.916949 25818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 14:46:27.922860 25818 server_base.cc:1034] running on GCE node
W20260928 14:46:27.922955 25860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:27.923141 25857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:27.922988 25858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:27.923573 25818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:46:27.923898 25818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:46:27.925104 25818 hybrid_clock.cc:648] HybridClock initialized: now 1790606787925080 us; error 42 us; skew 500 ppm
Sep 28 14:46:27 dist-test-slave-pv0w krb5kdc[24679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606787, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:46:27.927945 25818 init.cc:377] Logged in from keytab as kudu/127.9.33.65@KRBTEST.COM (short username kudu)
I20260928 14:46:27.928445 25818 webserver.cc:460] Webserver started at http://127.9.33.65:42967/ using document root <none> and password file <none>
I20260928 14:46:27.928700 25818 fs_manager.cc:359] Metadata directory not provided
I20260928 14:46:27.928771 25818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:46:27.930284 25818 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:46:27.931128 25867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:27.931357 25818 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:46:27.931468 25818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/wal
uuid: "eafe320ad6134d8397a349b3e0f30e54"
format_stamp: "Formatted at 2026-09-28 14:45:46 on dist-test-slave-pv0w"
server_key: "54693ff559a2d17a617d166c70065be0"
server_key_iv: "1f946ada8ebc254bf91cd9d9eaa6fa84"
server_key_version: "encryptionkey@0"
I20260928 14:46:27.931954 25818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:46:27.950911 25818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:46:27.953820 25818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:46:27.953987 25818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:46:27.954267 25818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:46:27.954838 25874 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260928 14:46:27.956647 25818 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260928 14:46:27.956707 25818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260928 14:46:27.956745 25818 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260928 14:46:27.957670 25818 ts_tablet_manager.cc:610] Registered 1 tablets
I20260928 14:46:27.957715 25818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260928 14:46:27.957839 25874 tablet_bootstrap.cc:492] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54: Bootstrap starting.
I20260928 14:46:27.965122 25818 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.65:41211
I20260928 14:46:27.965337 25982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.65:41211 every 8 connection(s)
I20260928 14:46:27.965631 25818 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
I20260928 14:46:27.968899 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 25818
I20260928 14:46:27.969305 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.66:46101
--local_ip_for_outbound_sockets=127.9.33.66
--tserver_master_addrs=127.9.33.126:37671
--webserver_port=35687
--webserver_interface=127.9.33.66
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:35845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:46:27.969923 25874 log.cc:824] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54: Log is configured to *not* fsync() on all Append() calls
I20260928 14:46:27.972672 25874 tablet_bootstrap.cc:492] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 14:46:27.973012 25874 tablet_bootstrap.cc:492] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54: Bootstrap complete.
I20260928 14:46:27.974067 25874 ts_tablet_manager.cc:1397] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54: Time spent bootstrapping tablet: real 0.016s user 0.015s sys 0.000s
I20260928 14:46:27.975399 25874 raft_consensus.cc:348] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86701327a744e368607ded503580579" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34371 } } peers { permanent_uuid: "41521aceb5454ef4977c9ffd7b683b6b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 46101 } } peers { permanent_uuid: "eafe320ad6134d8397a349b3e0f30e54" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41211 } }
I20260928 14:46:27.975705 25874 raft_consensus.cc:729] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: eafe320ad6134d8397a349b3e0f30e54, State: Initialized, Role: FOLLOWER
I20260928 14:46:27.975947 25874 consensus_queue.cc:260] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86701327a744e368607ded503580579" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34371 } } peers { permanent_uuid: "41521aceb5454ef4977c9ffd7b683b6b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 46101 } } peers { permanent_uuid: "eafe320ad6134d8397a349b3e0f30e54" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41211 } }
I20260928 14:46:27.976316 25874 ts_tablet_manager.cc:1428] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
W20260928 14:46:28.065277 25985 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:46:27.966427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.65:41401)
0928 14:46:27.966659 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:27.966717 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:27.967641 (+ 924us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:28.065061 (+ 97420us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":109,"threads_started":1}
W20260928 14:46:28.065543 25983 heartbeater.cc:644] Failed to heartbeat to 127.9.33.126:37671 (0 consecutive failures): Network error: Failed to ping master at 127.9.33.126:37671: Client connection negotiation failed: client connection to 127.9.33.126:37671: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260928 14:46:28.065889 25985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:28.065764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.65:55311)
0928 14:46:28.065820 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:28.065869 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260928 14:46:28.066274 25985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:28.066178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.65:43075)
0928 14:46:28.066232 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:28.066257 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260928 14:46:28.066393 25983 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260928 14:46:28.075151 25987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:46:28.075439 25987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:46:28.075482 25987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:46:28.076781 25987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:46:28.076858 25987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:46:28.076925 25987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.66
I20260928 14:46:28.078492 25987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:35845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.66:46101
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.33.66
--webserver_port=35687
--tserver_master_addrs=127.9.33.126:37671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.66
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:46:28.078770 25987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:46:28.079015 25987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:46:28.084231 25993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:28.084352 25987 server_base.cc:1034] running on GCE node
W20260928 14:46:28.084203 25994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:28.084196 25996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:28.084761 25987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:46:28.085042 25987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:46:28.086220 25987 hybrid_clock.cc:648] HybridClock initialized: now 1790606788086205 us; error 35 us; skew 500 ppm
Sep 28 14:46:28 dist-test-slave-pv0w krb5kdc[24679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606788, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:46:28.089021 25987 init.cc:377] Logged in from keytab as kudu/127.9.33.66@KRBTEST.COM (short username kudu)
I20260928 14:46:28.089469 25987 webserver.cc:460] Webserver started at http://127.9.33.66:35687/ using document root <none> and password file <none>
I20260928 14:46:28.089701 25987 fs_manager.cc:359] Metadata directory not provided
I20260928 14:46:28.089772 25987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:46:28.091166 25987 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:46:28.091989 26003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:28.092218 25987 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:46:28.092346 25987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/wal
uuid: "41521aceb5454ef4977c9ffd7b683b6b"
format_stamp: "Formatted at 2026-09-28 14:45:46 on dist-test-slave-pv0w"
server_key: "c0465e976428b8a74d6866d2b5916d33"
server_key_iv: "cdaf2e85d120be8e711704a8a720e34d"
server_key_version: "encryptionkey@0"
I20260928 14:46:28.092779 25987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:46:28.108167 25987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:46:28.110885 25987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:46:28.111083 25987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:46:28.111356 25987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:46:28.111943 26010 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260928 14:46:28.113545 25987 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260928 14:46:28.113629 25987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260928 14:46:28.113672 25987 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260928 14:46:28.114346 25987 ts_tablet_manager.cc:610] Registered 1 tablets
I20260928 14:46:28.114404 25987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260928 14:46:28.114532 26010 tablet_bootstrap.cc:492] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b: Bootstrap starting.
I20260928 14:46:28.121663 25987 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.66:46101
I20260928 14:46:28.121783 26117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.66:46101 every 8 connection(s)
I20260928 14:46:28.122118 25987 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
I20260928 14:46:28.123368 26120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:28.122795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.66:60943)
0928 14:46:28.123041 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:28.123148 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":73,"threads_started":1}
W20260928 14:46:28.123678 26118 heartbeater.cc:644] Failed to heartbeat to 127.9.33.126:37671 (0 consecutive failures): Network error: Failed to ping master at 127.9.33.126:37671: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
I20260928 14:46:28.124063 26120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:28.123926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.66:60843)
0928 14:46:28.123992 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:28.124026 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260928 14:46:28.124426 26120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:28.124334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.66:55493)
0928 14:46:28.124381 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:28.124406 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260928 14:46:28.124611 26118 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260928 14:46:28.125989 26010 log.cc:824] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b: Log is configured to *not* fsync() on all Append() calls
I20260928 14:46:28.127112 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 25987
I20260928 14:46:28.127470 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.67:34371
--local_ip_for_outbound_sockets=127.9.33.67
--tserver_master_addrs=127.9.33.126:37671
--webserver_port=36613
--webserver_interface=127.9.33.67
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:35845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:46:28.129444 26010 tablet_bootstrap.cc:492] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 14:46:28.129767 26010 tablet_bootstrap.cc:492] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b: Bootstrap complete.
I20260928 14:46:28.130911 26010 ts_tablet_manager.cc:1397] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b: Time spent bootstrapping tablet: real 0.017s user 0.011s sys 0.005s
I20260928 14:46:28.133316 26010 raft_consensus.cc:348] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86701327a744e368607ded503580579" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34371 } } peers { permanent_uuid: "41521aceb5454ef4977c9ffd7b683b6b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 46101 } } peers { permanent_uuid: "eafe320ad6134d8397a349b3e0f30e54" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41211 } }
I20260928 14:46:28.133864 26010 raft_consensus.cc:729] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41521aceb5454ef4977c9ffd7b683b6b, State: Initialized, Role: FOLLOWER
I20260928 14:46:28.134060 26010 consensus_queue.cc:260] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86701327a744e368607ded503580579" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34371 } } peers { permanent_uuid: "41521aceb5454ef4977c9ffd7b683b6b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 46101 } } peers { permanent_uuid: "eafe320ad6134d8397a349b3e0f30e54" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41211 } }
I20260928 14:46:28.134440 26010 ts_tablet_manager.cc:1428] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260928 14:46:28.134873 26120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:28.134703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.66:34571)
0928 14:46:28.134781 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:28.134829 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260928 14:46:28.135304 26120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:28.135177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.66:41087)
0928 14:46:28.135235 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:28.135270 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260928 14:46:28.216912 26122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:46:28.217128 26122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:46:28.217154 26122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:46:28.218273 26122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:46:28.218308 26122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:46:28.218335 26122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.67
I20260928 14:46:28.219785 26122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:35845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.67:34371
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.33.67
--webserver_port=36613
--tserver_master_addrs=127.9.33.126:37671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.67
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:46:28.219985 26122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:46:28.220158 26122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:46:28.225255 26131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:28.225278 26128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:28.225375 26122 server_base.cc:1034] running on GCE node
W20260928 14:46:28.225440 26129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:28.225641 26122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:46:28.225905 26122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:46:28.227092 26122 hybrid_clock.cc:648] HybridClock initialized: now 1790606788227086 us; error 37 us; skew 500 ppm
Sep 28 14:46:28 dist-test-slave-pv0w krb5kdc[24679](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606788, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:46:28.229750 26122 init.cc:377] Logged in from keytab as kudu/127.9.33.67@KRBTEST.COM (short username kudu)
I20260928 14:46:28.230247 26122 webserver.cc:460] Webserver started at http://127.9.33.67:36613/ using document root <none> and password file <none>
I20260928 14:46:28.230497 26122 fs_manager.cc:359] Metadata directory not provided
I20260928 14:46:28.230575 26122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:46:28.232055 26122 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:46:28.232723 26138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:28.232898 26122 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260928 14:46:28.233012 26122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/wal
uuid: "b86701327a744e368607ded503580579"
format_stamp: "Formatted at 2026-09-28 14:45:46 on dist-test-slave-pv0w"
server_key: "1745a03d9e9a59c21a864b52d9a5d2bd"
server_key_iv: "6296c5ef37a097596ab410c48d3b9a90"
server_key_version: "encryptionkey@0"
I20260928 14:46:28.233403 26122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:46:28.247603 26122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:46:28.250108 26122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:46:28.250273 26122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:46:28.250516 26122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:46:28.250851 26122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:46:28.250905 26122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:28.250976 26122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:46:28.251008 26122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:28.257237 26122 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.67:34371
I20260928 14:46:28.257277 26251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.67:34371 every 8 connection(s)
I20260928 14:46:28.257625 26122 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
I20260928 14:46:28.258805 26254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:28.258317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.67:38527)
0928 14:46:28.258570 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:28.258656 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":73,"threads_started":1}
W20260928 14:46:28.259212 26252 heartbeater.cc:644] Failed to heartbeat to 127.9.33.126:37671 (0 consecutive failures): Network error: Failed to ping master at 127.9.33.126:37671: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
I20260928 14:46:28.259510 26254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:28.259401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.67:35839)
0928 14:46:28.259455 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:28.259482 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260928 14:46:28.259846 26254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:28.259773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.67:33793)
0928 14:46:28.259802 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:28.259825 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260928 14:46:28.259979 26252 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260928 14:46:28.266049 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 26122
I20260928 14:46:29.067013 26256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:29.066764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.65:54147)
0928 14:46:29.066915 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:29.066946 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":73,"threads_started":1}
I20260928 14:46:29.136276 26257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:29.135948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.66:39607)
0928 14:46:29.136121 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:29.136159 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":112,"threads_started":1}
I20260928 14:46:29.260520 26258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:29.260304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.67:57623)
0928 14:46:29.260415 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:29.260457 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":55,"threads_started":1}
I20260928 14:46:29.667977 26259 raft_consensus.cc:482] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:46:29.668110 26259 raft_consensus.cc:504] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86701327a744e368607ded503580579" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34371 } } peers { permanent_uuid: "41521aceb5454ef4977c9ffd7b683b6b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 46101 } } peers { permanent_uuid: "eafe320ad6134d8397a349b3e0f30e54" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41211 } }
I20260928 14:46:29.668449 26259 leader_election.cc:288] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b86701327a744e368607ded503580579 (127.9.33.67:34371), 41521aceb5454ef4977c9ffd7b683b6b (127.9.33.66:46101)
I20260928 14:46:29.674736 26264 raft_consensus.cc:482] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:46:29.674873 26264 raft_consensus.cc:504] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86701327a744e368607ded503580579" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34371 } } peers { permanent_uuid: "41521aceb5454ef4977c9ffd7b683b6b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 46101 } } peers { permanent_uuid: "eafe320ad6134d8397a349b3e0f30e54" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41211 } }
I20260928 14:46:29.675271 26264 leader_election.cc:288] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b86701327a744e368607ded503580579 (127.9.33.67:34371), eafe320ad6134d8397a349b3e0f30e54 (127.9.33.65:41211)
Sep 28 14:46:29 dist-test-slave-pv0w krb5kdc[24679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606788, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for kudu/127.9.33.65@KRBTEST.COM
Sep 28 14:46:29 dist-test-slave-pv0w krb5kdc[24679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606787, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for kudu/127.9.33.66@KRBTEST.COM
Sep 28 14:46:29 dist-test-slave-pv0w krb5kdc[24679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606787, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for kudu/127.9.33.67@KRBTEST.COM
Sep 28 14:46:29 dist-test-slave-pv0w krb5kdc[24679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606788, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for kudu/127.9.33.67@KRBTEST.COM
I20260928 14:46:29.698002 26260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:29.668653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:46101 (local address 127.9.33.65:52397)
0928 14:46:29.668784 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:29.668795 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:29.668853 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:29.669434 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:29.669438 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:29.669948 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:46:29.670163 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:29.670169 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:29.671324 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:29.671325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:29.672263 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:29.672267 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:29.672978 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:29.672979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:29.673018 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:29.691354 (+ 18336us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:46:29.691370 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:46:29.693912 (+ 2542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:46:29.696398 (+ 2486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:46:29.696405 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:46:29.696407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:46:29.696686 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:46:29.696949 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:46:29.696950 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:46:29.696950 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:46:29.697077 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:46:29.697454 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:46:29.697461 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:46:29.697555 (+ 94us) client_negotiation.cc:769] Sending connection context
0928 14:46:29.697671 (+ 116us) client_negotiation.cc:240] Negotiation successful
0928 14:46:29.697854 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":93,"thread_start_us":59,"threads_started":1}
I20260928 14:46:29.698025 26263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:29.668839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:52397 (local address 127.9.33.66:46101)
0928 14:46:29.669074 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:29.669082 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:29.669102 (+ 20us) server_negotiation.cc:407] Connection header received
0928 14:46:29.669200 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:29.669202 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:29.669259 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:29.669372 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:29.670300 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:29.671194 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:29.672365 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:29.672787 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:29.672891 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:29.694025 (+ 21134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:29.694033 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:29.694033 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:46:29.694041 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:29.696285 (+ 2244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:29.696781 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:29.696784 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:29.696785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:29.696869 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:29.697138 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:29.697140 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:29.697140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:29.697390 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:29.697466 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:29.697751 (+ 285us) server_negotiation.cc:299] Negotiation successful
0928 14:46:29.697896 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":73,"threads_started":1}
I20260928 14:46:29.698484 26265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:29.675448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:41211 (local address 127.9.33.66:47989)
0928 14:46:29.675771 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:29.675790 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:29.675856 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:29.676111 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:29.676114 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:29.676605 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:46:29.676699 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:29.676702 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:29.678762 (+ 2060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:29.678762 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:29.679373 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:29.679375 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:29.679972 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:29.679973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:29.680041 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:29.691395 (+ 11354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:46:29.691418 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:46:29.693410 (+ 1992us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:46:29.696094 (+ 2684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:46:29.696100 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:46:29.696103 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:46:29.696419 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:46:29.696727 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:46:29.696728 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:46:29.696728 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:46:29.696857 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:46:29.698095 (+ 1238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:46:29.698102 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:46:29.698158 (+ 56us) client_negotiation.cc:769] Sending connection context
0928 14:46:29.698242 (+ 84us) client_negotiation.cc:240] Negotiation successful
0928 14:46:29.698360 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":3,"thread_start_us":62,"threads_started":1}
I20260928 14:46:29.698900 26268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:29.675626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:47989 (local address 127.9.33.65:41211)
0928 14:46:29.675814 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:29.675820 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:29.675837 (+ 17us) server_negotiation.cc:407] Connection header received
0928 14:46:29.675947 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:29.675952 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:29.675994 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:29.676071 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:29.677800 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:29.678681 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:29.679466 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:29.679880 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:29.679969 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:29.693527 (+ 13558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:29.693536 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:29.693536 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:46:29.693552 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:29.696015 (+ 2463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:29.696524 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:29.696528 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:29.696528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:29.696644 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:29.696931 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:29.696932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:29.696933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:29.697347 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:29.697439 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:29.698725 (+ 1286us) server_negotiation.cc:299] Negotiation successful
0928 14:46:29.698803 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":53,"threads_started":1}
I20260928 14:46:29.699038 26072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57f4cbaad13c4520a364de9f24910b1f" candidate_uuid: "eafe320ad6134d8397a349b3e0f30e54" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "41521aceb5454ef4977c9ffd7b683b6b" is_pre_election: true
I20260928 14:46:29.699175 26072 raft_consensus.cc:2454] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eafe320ad6134d8397a349b3e0f30e54 in term 1.
I20260928 14:46:29.699244 25937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57f4cbaad13c4520a364de9f24910b1f" candidate_uuid: "41521aceb5454ef4977c9ffd7b683b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "eafe320ad6134d8397a349b3e0f30e54" is_pre_election: true
I20260928 14:46:29.699366 25937 raft_consensus.cc:2454] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41521aceb5454ef4977c9ffd7b683b6b in term 1.
I20260928 14:46:29.699433 25869 leader_election.cc:302] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41521aceb5454ef4977c9ffd7b683b6b, eafe320ad6134d8397a349b3e0f30e54; no voters:
I20260928 14:46:29.699559 26261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:29.668651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:34371 (local address 127.9.33.65:53225)
0928 14:46:29.668768 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:29.668781 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:29.668843 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:29.669715 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:29.669716 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:29.669946 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:46:29.670163 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:29.670169 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:29.671286 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:29.671287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:29.672261 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:29.672263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:29.672885 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:29.672886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:29.673000 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:29.691354 (+ 18354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:46:29.691370 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:46:29.694356 (+ 2986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:46:29.697948 (+ 3592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:46:29.697949 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:46:29.697949 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:46:29.698155 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:46:29.698446 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:46:29.698447 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:46:29.698447 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:46:29.698509 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:46:29.699002 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:46:29.699003 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:46:29.699060 (+ 57us) client_negotiation.cc:769] Sending connection context
0928 14:46:29.699118 (+ 58us) client_negotiation.cc:240] Negotiation successful
0928 14:46:29.699213 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":8,"thread_start_us":53,"threads_started":1}
I20260928 14:46:29.699554 26259 raft_consensus.cc:2781] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260928 14:46:29.699599 26007 leader_election.cc:302] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41521aceb5454ef4977c9ffd7b683b6b, eafe320ad6134d8397a349b3e0f30e54; no voters:
I20260928 14:46:29.699633 26259 raft_consensus.cc:482] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:46:29.699633 26262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:29.668775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:53225 (local address 127.9.33.67:34371)
0928 14:46:29.668955 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:29.668964 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:29.668993 (+ 29us) server_negotiation.cc:407] Connection header received
0928 14:46:29.669517 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:29.669520 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:29.669569 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:29.669715 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:29.670326 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:29.671182 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:29.672384 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:29.672786 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:29.672917 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:29.694900 (+ 21983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:29.694915 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:29.694916 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:46:29.694932 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:29.697859 (+ 2927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:29.698253 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:29.698255 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:29.698255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:29.698365 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:29.698586 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:29.698587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:29.698588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:29.698909 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:29.699301 (+ 392us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:29.699380 (+ 79us) server_negotiation.cc:299] Negotiation successful
0928 14:46:29.699516 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":142,"thread_start_us":55,"threads_started":1}
I20260928 14:46:29.699661 26259 raft_consensus.cc:3037] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [term 1 FOLLOWER]: Advancing to term 2
I20260928 14:46:29.699772 26264 raft_consensus.cc:2781] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260928 14:46:29.699833 26264 raft_consensus.cc:482] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:46:29.699849 26264 raft_consensus.cc:3037] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b [term 1 FOLLOWER]: Advancing to term 2
I20260928 14:46:29.699970 26266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:29.675570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:34371 (local address 127.9.33.66:46003)
0928 14:46:29.675904 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:29.675914 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:29.675974 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:29.676176 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:29.676177 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:29.676627 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:46:29.676699 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:29.676702 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:29.677767 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:29.677768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:29.678611 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:29.678614 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:29.679034 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:29.679035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:29.679100 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:29.691402 (+ 12302us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:46:29.691407 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:46:29.694986 (+ 3579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:46:29.698001 (+ 3015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:46:29.698002 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:46:29.698002 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:46:29.698239 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:46:29.698464 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:46:29.698464 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:46:29.698465 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:46:29.698518 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:46:29.699228 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:46:29.699228 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:46:29.699273 (+ 45us) client_negotiation.cc:769] Sending connection context
0928 14:46:29.699342 (+ 69us) client_negotiation.cc:240] Negotiation successful
0928 14:46:29.699440 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":1,"thread_start_us":95,"threads_started":1}
W20260928 14:46:29.700367 25868 leader_election.cc:341] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b86701327a744e368607ded503580579 (127.9.33.67:34371): Not found: Tablet not found: 57f4cbaad13c4520a364de9f24910b1f
I20260928 14:46:29.701043 26259 raft_consensus.cc:504] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86701327a744e368607ded503580579" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34371 } } peers { permanent_uuid: "41521aceb5454ef4977c9ffd7b683b6b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 46101 } } peers { permanent_uuid: "eafe320ad6134d8397a349b3e0f30e54" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41211 } }
I20260928 14:46:29.701076 26264 raft_consensus.cc:504] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86701327a744e368607ded503580579" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34371 } } peers { permanent_uuid: "41521aceb5454ef4977c9ffd7b683b6b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 46101 } } peers { permanent_uuid: "eafe320ad6134d8397a349b3e0f30e54" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41211 } }
I20260928 14:46:29.701184 26259 leader_election.cc:288] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [CANDIDATE]: Term 2 election: Requested vote from peers b86701327a744e368607ded503580579 (127.9.33.67:34371), 41521aceb5454ef4977c9ffd7b683b6b (127.9.33.66:46101)
I20260928 14:46:29.701210 26264 leader_election.cc:288] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b [CANDIDATE]: Term 2 election: Requested vote from peers b86701327a744e368607ded503580579 (127.9.33.67:34371), eafe320ad6134d8397a349b3e0f30e54 (127.9.33.65:41211)
W20260928 14:46:29.701421 25868 leader_election.cc:341] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer b86701327a744e368607ded503580579 (127.9.33.67:34371): Not found: Tablet not found: 57f4cbaad13c4520a364de9f24910b1f
I20260928 14:46:29.701508 26267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:29.675587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:46003 (local address 127.9.33.67:34371)
0928 14:46:29.675859 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:29.675861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:29.676029 (+ 168us) server_negotiation.cc:407] Connection header received
0928 14:46:29.676052 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:29.676053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:29.676064 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:29.676121 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:29.676800 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:29.677658 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:29.678702 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:29.678953 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:29.679004 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:29.695093 (+ 16089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:29.695095 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:29.695095 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:46:29.695103 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:29.697914 (+ 2811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:29.698327 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:29.698328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:29.698329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:29.698392 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:29.698587 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:29.698588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:29.698588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:29.698906 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:29.699207 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:29.701307 (+ 2100us) server_negotiation.cc:299] Negotiation successful
0928 14:46:29.701392 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":79,"threads_started":1}
I20260928 14:46:29.701465 26072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57f4cbaad13c4520a364de9f24910b1f" candidate_uuid: "eafe320ad6134d8397a349b3e0f30e54" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "41521aceb5454ef4977c9ffd7b683b6b"
I20260928 14:46:29.701566 26072 raft_consensus.cc:2379] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate eafe320ad6134d8397a349b3e0f30e54 in current term 2: Already voted for candidate 41521aceb5454ef4977c9ffd7b683b6b in this term.
I20260928 14:46:29.701713 25937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57f4cbaad13c4520a364de9f24910b1f" candidate_uuid: "41521aceb5454ef4977c9ffd7b683b6b" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "eafe320ad6134d8397a349b3e0f30e54"
I20260928 14:46:29.701802 25937 raft_consensus.cc:2379] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 41521aceb5454ef4977c9ffd7b683b6b in current term 2: Already voted for candidate eafe320ad6134d8397a349b3e0f30e54 in this term.
W20260928 14:46:29.702056 26004 leader_election.cc:341] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b86701327a744e368607ded503580579 (127.9.33.67:34371): Not found: Tablet not found: 57f4cbaad13c4520a364de9f24910b1f
W20260928 14:46:29.702185 26004 leader_election.cc:341] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer b86701327a744e368607ded503580579 (127.9.33.67:34371): Not found: Tablet not found: 57f4cbaad13c4520a364de9f24910b1f
I20260928 14:46:29.702217 26004 leader_election.cc:302] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 41521aceb5454ef4977c9ffd7b683b6b; no voters: b86701327a744e368607ded503580579, eafe320ad6134d8397a349b3e0f30e54
I20260928 14:46:29.702294 26264 raft_consensus.cc:2726] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260928 14:46:29.702333 25869 leader_election.cc:302] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eafe320ad6134d8397a349b3e0f30e54; no voters: 41521aceb5454ef4977c9ffd7b683b6b, b86701327a744e368607ded503580579
I20260928 14:46:29.702418 26259 raft_consensus.cc:2726] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260928 14:46:30.067696 26261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:30.067549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.65:55389)
0928 14:46:30.067618 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:30.067655 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260928 14:46:30.137143 26266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:30.136937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.66:37691)
0928 14:46:30.137002 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:30.137071 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260928 14:46:30.261413 26269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:30.261181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.67:57349)
0928 14:46:30.261330 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:30.261371 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":63,"threads_started":1}
I20260928 14:46:31.068481 26270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:31.068213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.65:36959)
0928 14:46:31.068373 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:31.068416 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":88,"threads_started":1}
I20260928 14:46:31.138175 26271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:31.137850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.66:56991)
0928 14:46:31.138040 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:31.138098 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":78,"threads_started":1}
I20260928 14:46:31.262260 26272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:31.261971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.67:48189)
0928 14:46:31.262152 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:31.262192 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":78,"threads_started":1}
I20260928 14:46:31.474982 26273 raft_consensus.cc:482] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:46:31.475136 26273 raft_consensus.cc:504] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86701327a744e368607ded503580579" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34371 } } peers { permanent_uuid: "41521aceb5454ef4977c9ffd7b683b6b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 46101 } } peers { permanent_uuid: "eafe320ad6134d8397a349b3e0f30e54" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41211 } }
I20260928 14:46:31.475420 26273 leader_election.cc:288] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b86701327a744e368607ded503580579 (127.9.33.67:34371), 41521aceb5454ef4977c9ffd7b683b6b (127.9.33.66:46101)
I20260928 14:46:31.475832 26072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57f4cbaad13c4520a364de9f24910b1f" candidate_uuid: "eafe320ad6134d8397a349b3e0f30e54" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "41521aceb5454ef4977c9ffd7b683b6b" is_pre_election: true
I20260928 14:46:31.475977 26072 raft_consensus.cc:2454] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eafe320ad6134d8397a349b3e0f30e54 in term 2.
W20260928 14:46:31.475980 25868 leader_election.cc:341] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer b86701327a744e368607ded503580579 (127.9.33.67:34371): Not found: Tablet not found: 57f4cbaad13c4520a364de9f24910b1f
I20260928 14:46:31.476222 25869 leader_election.cc:302] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41521aceb5454ef4977c9ffd7b683b6b, eafe320ad6134d8397a349b3e0f30e54; no voters: b86701327a744e368607ded503580579
I20260928 14:46:31.476363 26273 raft_consensus.cc:2781] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260928 14:46:31.476411 26273 raft_consensus.cc:482] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:46:31.476429 26273 raft_consensus.cc:3037] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [term 2 FOLLOWER]: Advancing to term 3
I20260928 14:46:31.477458 26273 raft_consensus.cc:504] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86701327a744e368607ded503580579" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34371 } } peers { permanent_uuid: "41521aceb5454ef4977c9ffd7b683b6b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 46101 } } peers { permanent_uuid: "eafe320ad6134d8397a349b3e0f30e54" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41211 } }
I20260928 14:46:31.477656 26273 leader_election.cc:288] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [CANDIDATE]: Term 3 election: Requested vote from peers b86701327a744e368607ded503580579 (127.9.33.67:34371), 41521aceb5454ef4977c9ffd7b683b6b (127.9.33.66:46101)
W20260928 14:46:31.478039 25868 leader_election.cc:341] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [CANDIDATE]: Term 3 election: Tablet error from VoteRequest() call to peer b86701327a744e368607ded503580579 (127.9.33.67:34371): Not found: Tablet not found: 57f4cbaad13c4520a364de9f24910b1f
I20260928 14:46:31.477998 26072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57f4cbaad13c4520a364de9f24910b1f" candidate_uuid: "eafe320ad6134d8397a349b3e0f30e54" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "41521aceb5454ef4977c9ffd7b683b6b"
I20260928 14:46:31.478103 26072 raft_consensus.cc:3037] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b [term 2 FOLLOWER]: Advancing to term 3
I20260928 14:46:31.478986 26072 raft_consensus.cc:2454] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eafe320ad6134d8397a349b3e0f30e54 in term 3.
I20260928 14:46:31.479228 25869 leader_election.cc:302] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 41521aceb5454ef4977c9ffd7b683b6b, eafe320ad6134d8397a349b3e0f30e54; no voters: b86701327a744e368607ded503580579
I20260928 14:46:31.479342 26273 raft_consensus.cc:2781] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [term 3 FOLLOWER]: Leader election won for term 3
I20260928 14:46:31.479523 26273 raft_consensus.cc:686] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [term 3 LEADER]: Becoming Leader. State: Replica: eafe320ad6134d8397a349b3e0f30e54, State: Running, Role: LEADER
I20260928 14:46:31.479686 26273 consensus_queue.cc:237] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86701327a744e368607ded503580579" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34371 } } peers { permanent_uuid: "41521aceb5454ef4977c9ffd7b683b6b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 46101 } } peers { permanent_uuid: "eafe320ad6134d8397a349b3e0f30e54" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41211 } }
I20260928 14:46:31.479955 26270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:31.479835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.65:59773)
0928 14:46:31.479879 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:31.479911 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260928 14:46:31.885684 25868 consensus_peers.cc:487] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 -> Peer b86701327a744e368607ded503580579 (127.9.33.67:34371): Couldn't send request to peer b86701327a744e368607ded503580579. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 57f4cbaad13c4520a364de9f24910b1f. This is attempt 1: this message will repeat every 5th retry.
I20260928 14:46:31.957245 26072 raft_consensus.cc:1261] T 57f4cbaad13c4520a364de9f24910b1f P 41521aceb5454ef4977c9ffd7b683b6b [term 3 FOLLOWER]: Refusing update from remote peer eafe320ad6134d8397a349b3e0f30e54: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260928 14:46:31.957659 26273 consensus_queue.cc:1035] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41521aceb5454ef4977c9ffd7b683b6b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 46101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260928 14:46:31.958784 26278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:31.958606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.66:58529)
0928 14:46:31.958702 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:31.958740 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78,"thread_start_us":47,"threads_started":1}
I20260928 14:46:31.959884 26270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:31.959746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.65:43467)
0928 14:46:31.959790 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:31.959829 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260928 14:46:31.960095 26278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:31.960003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.66:37569)
0928 14:46:31.960042 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:31.960070 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260928 14:46:32.263088 26281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:32.262837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.67:46831)
0928 14:46:32.262995 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:32.263039 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":66,"threads_started":1}
I20260928 14:46:32.421386 26282 ts_tablet_manager.cc:927] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579: Initiating tablet copy from peer eafe320ad6134d8397a349b3e0f30e54 (127.9.33.65:41211)
I20260928 14:46:32.421622 26282 tablet_copy_client.cc:316] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579: tablet copy: Beginning tablet copy session from remote peer at address 127.9.33.65:41211
Sep 28 14:46:32 dist-test-slave-pv0w krb5kdc[24679](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606788, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for kudu/127.9.33.65@KRBTEST.COM
I20260928 14:46:32.430974 26284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:32.422008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:52283 (local address 127.9.33.65:41211)
0928 14:46:32.422186 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:32.422188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:32.422196 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:46:32.422220 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:32.422220 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:32.422229 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:32.422292 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:32.422884 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:32.423385 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:32.424326 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:32.424533 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:32.424599 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:32.427074 (+ 2475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:32.427076 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:32.427076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:46:32.427085 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:32.429047 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:32.429708 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:32.429710 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:32.429710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:32.429877 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:32.430201 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:32.430203 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:32.430203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:32.430370 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:32.430532 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:32.430752 (+ 220us) server_negotiation.cc:299] Negotiation successful
0928 14:46:32.430853 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":111,"threads_started":1}
I20260928 14:46:32.430994 26281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:32.421889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:41211 (local address 127.9.33.67:52283)
0928 14:46:32.421984 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:32.422008 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:32.422095 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:32.422336 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:32.422344 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:32.422700 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:46:32.422776 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:32.422778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:32.423479 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:32.423480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:32.424232 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:32.424234 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:32.424615 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:32.424616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:32.424679 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:32.425263 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:46:32.425268 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:46:32.426899 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:46:32.429188 (+ 2289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:46:32.429202 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:46:32.429205 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:46:32.429532 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:46:32.430050 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:46:32.430052 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:46:32.430052 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:46:32.430119 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:46:32.430539 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:46:32.430551 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:46:32.430613 (+ 62us) client_negotiation.cc:769] Sending connection context
0928 14:46:32.430721 (+ 108us) client_negotiation.cc:240] Negotiation successful
0928 14:46:32.430862 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260928 14:46:32.431492 25957 tablet_copy_service.cc:140] P eafe320ad6134d8397a349b3e0f30e54: Received BeginTabletCopySession request for tablet 57f4cbaad13c4520a364de9f24910b1f from peer b86701327a744e368607ded503580579 ({username='kudu', principal='kudu/127.9.33.67@KRBTEST.COM'} at 127.9.33.67:52283)
I20260928 14:46:32.431583 25957 tablet_copy_service.cc:161] P eafe320ad6134d8397a349b3e0f30e54: Beginning new tablet copy session on tablet 57f4cbaad13c4520a364de9f24910b1f from peer b86701327a744e368607ded503580579 at {username='kudu', principal='kudu/127.9.33.67@KRBTEST.COM'} at 127.9.33.67:52283: session id = b86701327a744e368607ded503580579-57f4cbaad13c4520a364de9f24910b1f
I20260928 14:46:32.432204 25957 tablet_copy_source_session.cc:205] T 57f4cbaad13c4520a364de9f24910b1f P eafe320ad6134d8397a349b3e0f30e54: Tablet Copy: opened 0 blocks and 1 log segments
I20260928 14:46:32.433502 26282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57f4cbaad13c4520a364de9f24910b1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:46:32.436510 26282 tablet_copy_client.cc:791] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579: tablet copy: Starting download of 0 data blocks...
I20260928 14:46:32.436691 26282 tablet_copy_client.cc:655] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579: tablet copy: Starting download of 1 WAL segments...
I20260928 14:46:32.437819 26282 tablet_copy_client.cc:531] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260928 14:46:32.438634 26282 tablet_bootstrap.cc:492] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579: Bootstrap starting.
I20260928 14:46:32.440551 26282 log.cc:824] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579: Log is configured to *not* fsync() on all Append() calls
I20260928 14:46:32.442644 26282 tablet_bootstrap.cc:492] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260928 14:46:32.442888 26282 tablet_bootstrap.cc:492] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579: Bootstrap complete.
I20260928 14:46:32.443045 26282 ts_tablet_manager.cc:1397] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260928 14:46:32.444181 26282 raft_consensus.cc:348] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86701327a744e368607ded503580579" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34371 } } peers { permanent_uuid: "41521aceb5454ef4977c9ffd7b683b6b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 46101 } } peers { permanent_uuid: "eafe320ad6134d8397a349b3e0f30e54" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41211 } }
I20260928 14:46:32.444362 26282 raft_consensus.cc:729] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: b86701327a744e368607ded503580579, State: Initialized, Role: FOLLOWER
I20260928 14:46:32.444533 26282 consensus_queue.cc:260] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b86701327a744e368607ded503580579" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34371 } } peers { permanent_uuid: "41521aceb5454ef4977c9ffd7b683b6b" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 46101 } } peers { permanent_uuid: "eafe320ad6134d8397a349b3e0f30e54" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41211 } }
I20260928 14:46:32.444849 26282 ts_tablet_manager.cc:1428] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20260928 14:46:32.445228 26281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:32.445106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.67:37311)
0928 14:46:32.445162 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:32.445201 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260928 14:46:32.445217 25957 tablet_copy_service.cc:342] P eafe320ad6134d8397a349b3e0f30e54: Request end of tablet copy session b86701327a744e368607ded503580579-57f4cbaad13c4520a364de9f24910b1f received from {username='kudu', principal='kudu/127.9.33.67@KRBTEST.COM'} at 127.9.33.67:52283
I20260928 14:46:32.445288 25957 tablet_copy_service.cc:433] P eafe320ad6134d8397a349b3e0f30e54: ending tablet copy session b86701327a744e368607ded503580579-57f4cbaad13c4520a364de9f24910b1f on tablet 57f4cbaad13c4520a364de9f24910b1f with peer b86701327a744e368607ded503580579
I20260928 14:46:32.445679 26281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:32.445495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.67:59999)
0928 14:46:32.445596 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:32.445627 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260928 14:46:32.919409 26289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:32.919184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.65:42763)
0928 14:46:32.919316 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:32.919361 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":63,"threads_started":1}
I20260928 14:46:32.919370 26205 raft_consensus.cc:1203] T 57f4cbaad13c4520a364de9f24910b1f P b86701327a744e368607ded503580579 [term 3 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[3.3-3.3] Dedup: 3.3->[]
I20260928 14:46:32.919921 26281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:32.919816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.67:49925)
0928 14:46:32.919853 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:32.919887 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260928 14:46:32.960961 26290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:32.960702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.66:44819)
0928 14:46:32.960845 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:32.960896 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":79,"threads_started":1}
I20260928 14:46:33.920603 26291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:33.920288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.65:57737)
0928 14:46:33.920454 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:33.920513 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":80,"threads_started":1}
I20260928 14:46:33.920899 26292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:33.920542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.67:32907)
0928 14:46:33.920788 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:33.920841 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":75,"threads_started":1}
I20260928 14:46:33.921157 26291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:33.920935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.65:42297)
0928 14:46:33.921063 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:33.921121 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260928 14:46:33.962498 26293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:33.962123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.66:38389)
0928 14:46:33.962359 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:33.962418 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":104,"threads_started":1}
I20260928 14:46:34.922132 26294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:34.921731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.67:41465)
0928 14:46:34.921981 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:34.922047 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":100,"threads_started":1}
I20260928 14:46:34.922169 26295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:34.921888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.65:47817)
0928 14:46:34.922074 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:34.922117 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":113,"threads_started":1}
I20260928 14:46:34.963649 26296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:34.963259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.66:34479)
0928 14:46:34.963504 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:34.963563 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":125,"threads_started":1}
I20260928 14:46:35.923089 26298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:35.922808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.67:47311)
0928 14:46:35.922969 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:35.923018 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":93,"threads_started":1}
I20260928 14:46:35.923081 26297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:35.922810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.65:54033)
0928 14:46:35.922999 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:35.923033 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":103,"threads_started":1}
I20260928 14:46:35.964512 26299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:35.964241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.66:57315)
0928 14:46:35.964408 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:35.964455 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":85,"threads_started":1}
I20260928 14:46:36.923863 26300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:36.923615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.65:44067)
0928 14:46:36.923778 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:36.923813 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":82,"threads_started":1}
I20260928 14:46:36.923877 26301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:36.923610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.67:36903)
0928 14:46:36.923802 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:36.923833 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":66,"threads_started":1}
I20260928 14:46:36.965355 26302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:36.965114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.66:38585)
0928 14:46:36.965272 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:36.965307 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":74,"threads_started":1}
I20260928 14:46:37.924621 26304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:37.924391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.67:42301)
0928 14:46:37.924530 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:37.924573 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":83,"threads_started":1}
I20260928 14:46:37.924649 26305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:37.924391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.65:59569)
0928 14:46:37.924545 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:37.924605 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":84,"threads_started":1}
I20260928 14:46:37.966190 26306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:37.965915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.66:36903)
0928 14:46:37.966074 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:37.966119 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":76,"threads_started":1}
I20260928 14:46:38.925519 26308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:38.925251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.67:43031)
0928 14:46:38.925425 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:38.925464 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":105,"threads_started":1}
I20260928 14:46:38.925519 26309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:38.925251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.65:52243)
0928 14:46:38.925425 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:38.925464 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":84,"threads_started":1}
I20260928 14:46:38.967038 26310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:38.966801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.66:34373)
0928 14:46:38.966949 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:38.966997 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":75,"threads_started":1}
I20260928 14:46:39.926255 26311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:39.926023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.65:52087)
0928 14:46:39.926171 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:39.926207 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":61,"threads_started":1}
I20260928 14:46:39.926256 26312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:39.926033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.67:33067)
0928 14:46:39.926177 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:39.926214 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":79,"threads_started":1}
I20260928 14:46:39.967837 26313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:39.967565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.66:40103)
0928 14:46:39.967721 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:39.967781 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":55,"threads_started":1}
I20260928 14:46:40.926988 26315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:40.926756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.65:41969)
0928 14:46:40.926912 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:40.926946 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":96,"threads_started":1}
I20260928 14:46:40.926981 26314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:40.926768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.67:40461)
0928 14:46:40.926893 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:40.926933 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":80,"threads_started":1}
I20260928 14:46:40.968532 26316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:40.968301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.66:38647)
0928 14:46:40.968447 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:40.968489 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":60,"threads_started":1}
I20260928 14:46:41.927821 26318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:41.927525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.65:33287)
0928 14:46:41.927691 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:41.927764 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":83,"threads_started":1}
I20260928 14:46:41.927831 26317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:41.927522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.67:46457)
0928 14:46:41.927687 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:41.927784 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":80,"threads_started":1}
I20260928 14:46:41.969269 26319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:41.969046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.66:47449)
0928 14:46:41.969184 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:41.969225 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":60,"threads_started":1}
I20260928 14:46:42.928637 26320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:42.928391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.65:41775)
0928 14:46:42.928546 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:42.928589 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":73,"threads_started":1}
I20260928 14:46:42.928637 26321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:42.928391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.67:44213)
0928 14:46:42.928548 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:42.928590 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":75,"threads_started":1}
I20260928 14:46:42.970198 26322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:42.969915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37671 (local address 127.9.33.66:54435)
0928 14:46:42.970095 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:42.970137 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.126:37671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":89,"threads_started":1}
I20260928 14:46:43.273488 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 25818
I20260928 14:46:43.278671 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 25987
I20260928 14:46:43.283424 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 26122
2026-09-28T14:46:43Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58546 ms)
[----------] 30 tests from SecurityITest (241746 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:46:43 dist-test-slave-pv0w krb5kdc[26325](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:46:43 dist-test-slave-pv0w krb5kdc[26325](info): set up 2 sockets
Sep 28 14:46:43 dist-test-slave-pv0w krb5kdc[26325](info): commencing operation
krb5kdc: starting...
W20260928 14:46:44.340282 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:46:44 dist-test-slave-pv0w krb5kdc[26325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606804, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:46:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:46:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.33.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
I20260928 14:46:44.467643 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:33077
--webserver_interface=127.9.33.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:37831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:33077
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:46:44.566536 26338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:46:44.566944 26338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:46:44.567010 26338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:46:44.569461 26338 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:46:44.569591 26338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:46:44.569639 26338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:46:44.569686 26338 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:46:44.569707 26338 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:46:44.572856 26338 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:37831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:33077
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:33077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:46:44.573184 26338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:46:44.573458 26338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:46:44.579381 26346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:44.579409 26343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:44.579525 26338 server_base.cc:1034] running on GCE node
W20260928 14:46:44.579409 26344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:44.580036 26338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:46:44.580593 26338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:46:44.581827 26338 hybrid_clock.cc:648] HybridClock initialized: now 1790606804581787 us; error 46 us; skew 500 ppm
Sep 28 14:46:44 dist-test-slave-pv0w krb5kdc[26325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606804, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:46:44.585081 26338 init.cc:377] Logged in from keytab as kudu/127.9.33.126@KRBTEST.COM (short username kudu)
I20260928 14:46:44.585754 26338 webserver.cc:460] Webserver started at http://127.9.33.126:33445/ using document root <none> and password file <none>
I20260928 14:46:44.586083 26338 fs_manager.cc:359] Metadata directory not provided
I20260928 14:46:44.586202 26338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:46:44.586365 26338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:46:44.588135 26338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "d2b8b32cd503484ba11cba08a8ec588e"
format_stamp: "Formatted at 2026-09-28 14:46:44 on dist-test-slave-pv0w"
server_key: "78b4c9ef64de32e95a80962d52e69d7c"
server_key_iv: "ee7533e25c242f6a02251fc447b7cb77"
server_key_version: "encryptionkey@0"
I20260928 14:46:44.588590 26338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "d2b8b32cd503484ba11cba08a8ec588e"
format_stamp: "Formatted at 2026-09-28 14:46:44 on dist-test-slave-pv0w"
server_key: "78b4c9ef64de32e95a80962d52e69d7c"
server_key_iv: "ee7533e25c242f6a02251fc447b7cb77"
server_key_version: "encryptionkey@0"
I20260928 14:46:44.590731 26338 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260928 14:46:44.591792 26353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:44.592002 26338 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:46:44.592125 26338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "d2b8b32cd503484ba11cba08a8ec588e"
format_stamp: "Formatted at 2026-09-28 14:46:44 on dist-test-slave-pv0w"
server_key: "78b4c9ef64de32e95a80962d52e69d7c"
server_key_iv: "ee7533e25c242f6a02251fc447b7cb77"
server_key_version: "encryptionkey@0"
I20260928 14:46:44.592217 26338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:46:44.612854 26338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:46:44.615293 26338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:46:44.615414 26338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:46:44.619469 26338 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:33077
I20260928 14:46:44.619537 26405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:33077 every 8 connection(s)
I20260928 14:46:44.619953 26338 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:46:44.620931 26406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:46:44.623373 26406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2b8b32cd503484ba11cba08a8ec588e: Bootstrap starting.
I20260928 14:46:44.623389 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 26338
I20260928 14:46:44.623525 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:46:44.623857 9349 external_mini_cluster.cc:1428] Setting key 529ee3c54ef418c370aabc0778ccb756
I20260928 14:46:44.624128 26406 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2b8b32cd503484ba11cba08a8ec588e: Neither blocks nor log segments found. Creating new log.
I20260928 14:46:44.624481 26406 log.cc:824] T 00000000000000000000000000000000 P d2b8b32cd503484ba11cba08a8ec588e: Log is configured to *not* fsync() on all Append() calls
I20260928 14:46:44.625263 26406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2b8b32cd503484ba11cba08a8ec588e: No bootstrap required, opened a new log
I20260928 14:46:44.627816 26406 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2b8b32cd503484ba11cba08a8ec588e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2b8b32cd503484ba11cba08a8ec588e" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 33077 } }
I20260928 14:46:44.628019 26406 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2b8b32cd503484ba11cba08a8ec588e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:46:44.628058 26406 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2b8b32cd503484ba11cba08a8ec588e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2b8b32cd503484ba11cba08a8ec588e, State: Initialized, Role: FOLLOWER
I20260928 14:46:44.628189 26406 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2b8b32cd503484ba11cba08a8ec588e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2b8b32cd503484ba11cba08a8ec588e" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 33077 } }
I20260928 14:46:44.628260 26406 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2b8b32cd503484ba11cba08a8ec588e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:46:44.628288 26406 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2b8b32cd503484ba11cba08a8ec588e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:46:44.628336 26406 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2b8b32cd503484ba11cba08a8ec588e [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:46:44.628964 26406 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2b8b32cd503484ba11cba08a8ec588e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2b8b32cd503484ba11cba08a8ec588e" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 33077 } }
I20260928 14:46:44.629109 26406 leader_election.cc:302] T 00000000000000000000000000000000 P d2b8b32cd503484ba11cba08a8ec588e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2b8b32cd503484ba11cba08a8ec588e; no voters:
I20260928 14:46:44.629307 26406 leader_election.cc:288] T 00000000000000000000000000000000 P d2b8b32cd503484ba11cba08a8ec588e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:46:44.629423 26411 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2b8b32cd503484ba11cba08a8ec588e [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:46:44.629595 26406 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2b8b32cd503484ba11cba08a8ec588e [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:46:44.629629 26411 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2b8b32cd503484ba11cba08a8ec588e [term 1 LEADER]: Becoming Leader. State: Replica: d2b8b32cd503484ba11cba08a8ec588e, State: Running, Role: LEADER
I20260928 14:46:44.629772 26411 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2b8b32cd503484ba11cba08a8ec588e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2b8b32cd503484ba11cba08a8ec588e" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 33077 } }
I20260928 14:46:44.631199 26412 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2b8b32cd503484ba11cba08a8ec588e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2b8b32cd503484ba11cba08a8ec588e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2b8b32cd503484ba11cba08a8ec588e" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 33077 } } }
I20260928 14:46:44.631290 26413 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2b8b32cd503484ba11cba08a8ec588e [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2b8b32cd503484ba11cba08a8ec588e. Latest consensus state: current_term: 1 leader_uuid: "d2b8b32cd503484ba11cba08a8ec588e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2b8b32cd503484ba11cba08a8ec588e" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 33077 } } }
I20260928 14:46:44.631506 26412 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2b8b32cd503484ba11cba08a8ec588e [sys.catalog]: This master's current role is: LEADER
I20260928 14:46:44.631518 26413 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2b8b32cd503484ba11cba08a8ec588e [sys.catalog]: This master's current role is: LEADER
Sep 28 14:46:44 dist-test-slave-pv0w krb5kdc[26325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606804, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
W20260928 14:46:44.631845 26426 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d2b8b32cd503484ba11cba08a8ec588e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:46:44.631909 26426 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:46:44.631975 26428 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:46:44.632160 26428 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:46:44.634495 26428 catalog_manager.cc:1347] Generated new cluster ID: 1c172a15e62743b6bc60fc8fa346802b
I20260928 14:46:44.634554 26428 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:46:44.636183 26409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:44.624484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57236 (local address 127.9.33.126:33077)
0928 14:46:44.624733 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:44.624744 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:44.624780 (+ 36us) server_negotiation.cc:407] Connection header received
0928 14:46:44.625323 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:44.625331 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:44.625420 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:44.625576 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:44.626418 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:44.627359 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:44.628507 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:44.628958 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:44.629041 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:44.632246 (+ 3205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:44.632250 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:44.632251 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:46:44.632262 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:44.634330 (+ 2068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:44.634792 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:44.634795 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:44.634796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:44.634912 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:44.635126 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:44.635127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:44.635127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:44.635483 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:44.635676 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:44.635794 (+ 118us) server_negotiation.cc:299] Negotiation successful
0928 14:46:44.635955 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":97,"threads_started":1}
I20260928 14:46:44.658775 26428 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:46:44.659574 26428 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:46:44.664955 26428 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2b8b32cd503484ba11cba08a8ec588e: Generated new TSK 0
I20260928 14:46:44.665195 26428 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.33.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab.
Entry for principal kudu/127.9.33.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab.
I20260928 14:46:44.695201 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.65:0
--local_ip_for_outbound_sockets=127.9.33.65
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:33077
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:37831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:46:44.782819 26432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:46:44.783146 26432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:46:44.783190 26432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:46:44.784593 26432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:46:44.784654 26432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:46:44.784685 26432 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.65
I20260928 14:46:44.786296 26432 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:37831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:33077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.65
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:46:44.786576 26432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:46:44.786862 26432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:46:44.792354 26438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:44.792531 26432 server_base.cc:1034] running on GCE node
W20260928 14:46:44.792354 26440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:44.792387 26437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:44.792871 26432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:46:44.793154 26432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:46:44.794335 26432 hybrid_clock.cc:648] HybridClock initialized: now 1790606804794312 us; error 38 us; skew 500 ppm
Sep 28 14:46:44 dist-test-slave-pv0w krb5kdc[26325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606804, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:46:44.797250 26432 init.cc:377] Logged in from keytab as kudu/127.9.33.65@KRBTEST.COM (short username kudu)
I20260928 14:46:44.797770 26432 webserver.cc:460] Webserver started at http://127.9.33.65:35551/ using document root <none> and password file <none>
I20260928 14:46:44.798032 26432 fs_manager.cc:359] Metadata directory not provided
I20260928 14:46:44.798107 26432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:46:44.798262 26432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:46:44.799508 26432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-0/data/instance:
uuid: "84b32c8c7b5042b3a4f24fd4b6e01af6"
format_stamp: "Formatted at 2026-09-28 14:46:44 on dist-test-slave-pv0w"
server_key: "4b28fd1cd05e4f7baba603b9af93bbe1"
server_key_iv: "cc3eb877b5ed604ebe85254b874508c5"
server_key_version: "encryptionkey@0"
I20260928 14:46:44.799929 26432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance:
uuid: "84b32c8c7b5042b3a4f24fd4b6e01af6"
format_stamp: "Formatted at 2026-09-28 14:46:44 on dist-test-slave-pv0w"
server_key: "4b28fd1cd05e4f7baba603b9af93bbe1"
server_key_iv: "cc3eb877b5ed604ebe85254b874508c5"
server_key_version: "encryptionkey@0"
I20260928 14:46:44.801348 26432 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:46:44.802203 26447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:44.802379 26432 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:46:44.802488 26432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-0/wal
uuid: "84b32c8c7b5042b3a4f24fd4b6e01af6"
format_stamp: "Formatted at 2026-09-28 14:46:44 on dist-test-slave-pv0w"
server_key: "4b28fd1cd05e4f7baba603b9af93bbe1"
server_key_iv: "cc3eb877b5ed604ebe85254b874508c5"
server_key_version: "encryptionkey@0"
I20260928 14:46:44.802598 26432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:46:44.813776 26432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:46:44.816079 26432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:46:44.816253 26432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:46:44.816501 26432 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:46:44.816846 26432 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:46:44.816903 26432 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:44.816946 26432 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:46:44.816995 26432 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:44.822993 26432 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.65:44171
I20260928 14:46:44.823112 26560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.65:44171 every 8 connection(s)
I20260928 14:46:44.823421 26432 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
Sep 28 14:46:44 dist-test-slave-pv0w krb5kdc[26325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606804, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:46:44.830796 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 26432
I20260928 14:46:44.830924 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance
I20260928 14:46:44.831142 9349 external_mini_cluster.cc:1428] Setting key 6102d736fa746551818c299385b991cb
I20260928 14:46:44.834052 26409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:44.824308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:58339 (local address 127.9.33.126:33077)
0928 14:46:44.824380 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:44.824383 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:44.824653 (+ 270us) server_negotiation.cc:407] Connection header received
0928 14:46:44.825138 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:44.825139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:44.825150 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:44.825204 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:44.826189 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:44.826702 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:44.827815 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:44.828024 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:44.828085 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:44.830433 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:44.830434 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:44.830434 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:46:44.830448 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:44.832213 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:44.832769 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:44.832770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:44.832771 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:44.832833 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:44.833174 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:44.833176 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:44.833176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:44.833363 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:44.833458 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:44.833831 (+ 373us) server_negotiation.cc:299] Negotiation successful
0928 14:46:44.833935 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260928 14:46:44.834128 26563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:44.824283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:33077 (local address 127.9.33.65:58339)
0928 14:46:44.824486 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:44.824563 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:44.825047 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:44.825264 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:44.825267 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:44.825863 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:46:44.826066 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:44.826071 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:44.826795 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:44.826796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:44.827637 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:44.827640 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:44.828109 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:44.828110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:44.828225 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:44.828831 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:46:44.828840 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:46:44.830345 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:46:44.832301 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:46:44.832312 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:46:44.832315 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:46:44.832672 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:46:44.832911 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:46:44.832912 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:46:44.832912 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:46:44.833076 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:46:44.833454 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:46:44.833462 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:46:44.833668 (+ 206us) client_negotiation.cc:769] Sending connection context
0928 14:46:44.833788 (+ 120us) client_negotiation.cc:240] Negotiation successful
0928 14:46:44.833927 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":105,"threads_started":1}
I20260928 14:46:44.834673 26561 heartbeater.cc:347] Connected to a master server at 127.9.33.126:33077
I20260928 14:46:44.834777 26561 heartbeater.cc:464] Registering TS with master...
I20260928 14:46:44.834972 26561 heartbeater.cc:511] Master 127.9.33.126:33077 requested a full tablet report, sending...
I20260928 14:46:44.835556 26370 ts_manager.cc:194] Registered new tserver with Master: 84b32c8c7b5042b3a4f24fd4b6e01af6 (127.9.33.65:44171)
I20260928 14:46:44.836223 26370 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.65@KRBTEST.COM'} at 127.9.33.65:58339
WARNING: no policy specified for kudu/127.9.33.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab.
Entry for principal kudu/127.9.33.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab.
I20260928 14:46:44.859709 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.66:0
--local_ip_for_outbound_sockets=127.9.33.66
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:33077
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:37831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:46:44.942135 26566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:46:44.942350 26566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:46:44.942375 26566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:46:44.943575 26566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:46:44.943624 26566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:46:44.943652 26566 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.66
I20260928 14:46:44.945209 26566 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:37831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:33077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.66
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:46:44.945487 26566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:46:44.945739 26566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:46:44.950855 26571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:44.950879 26572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:44.950861 26574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:44.951220 26566 server_base.cc:1034] running on GCE node
I20260928 14:46:44.951491 26566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:46:44.951817 26566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:46:44.952993 26566 hybrid_clock.cc:648] HybridClock initialized: now 1790606804952971 us; error 39 us; skew 500 ppm
Sep 28 14:46:44 dist-test-slave-pv0w krb5kdc[26325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606804, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:46:44.955673 26566 init.cc:377] Logged in from keytab as kudu/127.9.33.66@KRBTEST.COM (short username kudu)
I20260928 14:46:44.956224 26566 webserver.cc:460] Webserver started at http://127.9.33.66:40289/ using document root <none> and password file <none>
I20260928 14:46:44.956473 26566 fs_manager.cc:359] Metadata directory not provided
I20260928 14:46:44.956518 26566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:46:44.956652 26566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:46:44.957732 26566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-1/data/instance:
uuid: "adbfb2890c9945538f3ff3b640cff210"
format_stamp: "Formatted at 2026-09-28 14:46:44 on dist-test-slave-pv0w"
server_key: "ea9fadf5d1f0891c42edc3a9be4a12a9"
server_key_iv: "8d1aab69d391773a09c003a244b93d4b"
server_key_version: "encryptionkey@0"
I20260928 14:46:44.958091 26566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance:
uuid: "adbfb2890c9945538f3ff3b640cff210"
format_stamp: "Formatted at 2026-09-28 14:46:44 on dist-test-slave-pv0w"
server_key: "ea9fadf5d1f0891c42edc3a9be4a12a9"
server_key_iv: "8d1aab69d391773a09c003a244b93d4b"
server_key_version: "encryptionkey@0"
I20260928 14:46:44.959416 26566 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:46:44.960280 26581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:44.960466 26566 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:44.960570 26566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-1/wal
uuid: "adbfb2890c9945538f3ff3b640cff210"
format_stamp: "Formatted at 2026-09-28 14:46:44 on dist-test-slave-pv0w"
server_key: "ea9fadf5d1f0891c42edc3a9be4a12a9"
server_key_iv: "8d1aab69d391773a09c003a244b93d4b"
server_key_version: "encryptionkey@0"
I20260928 14:46:44.960686 26566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:46:44.972981 26566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:46:44.975299 26566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:46:44.975483 26566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:46:44.975798 26566 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:46:44.976125 26566 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:46:44.976181 26566 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:44.976226 26566 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:46:44.976255 26566 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:44.982307 26566 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.66:34857
I20260928 14:46:44.982352 26694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.66:34857 every 8 connection(s)
I20260928 14:46:44.982689 26566 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
I20260928 14:46:44.984807 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 26566
I20260928 14:46:44.984929 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance
I20260928 14:46:44.985123 9349 external_mini_cluster.cc:1428] Setting key c0b587dffbdaa33668c7e98394603883
Sep 28 14:46:44 dist-test-slave-pv0w krb5kdc[26325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606804, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.33.67@KRBTEST.COM; defaulting to no policy
I20260928 14:46:44.993928 26409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:44.983454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:39809 (local address 127.9.33.126:33077)
0928 14:46:44.983508 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:44.983511 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:44.983772 (+ 261us) server_negotiation.cc:407] Connection header received
0928 14:46:44.984319 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:44.984320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:44.984329 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:44.984392 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:44.985558 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:44.986282 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:44.987333 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:44.987553 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:44.987635 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:44.990476 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:44.990479 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:44.990479 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:46:44.990488 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:44.992302 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:44.992809 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:44.992810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:44.992811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:44.992876 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:44.993166 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:44.993167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:44.993167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:44.993306 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:44.993388 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:44.993733 (+ 345us) server_negotiation.cc:299] Negotiation successful
0928 14:46:44.993829 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260928 14:46:44.994167 26697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:44.983401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:33077 (local address 127.9.33.66:39809)
0928 14:46:44.983567 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:44.983660 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:44.984224 (+ 564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:44.984452 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:44.984455 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:44.985126 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:46:44.985452 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:44.985459 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:44.986385 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:44.986386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:44.987229 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:44.987232 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:44.987661 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:44.987663 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:44.987868 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:44.988590 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:46:44.988602 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:46:44.990353 (+ 1751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:46:44.992422 (+ 2069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:46:44.992434 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:46:44.992437 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:46:44.992708 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:46:44.992962 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:46:44.992963 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:46:44.992963 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:46:44.993090 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:46:44.993378 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:46:44.993386 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:46:44.993584 (+ 198us) client_negotiation.cc:769] Sending connection context
0928 14:46:44.993708 (+ 124us) client_negotiation.cc:240] Negotiation successful
0928 14:46:44.993893 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":95,"threads_started":1}
I20260928 14:46:44.994812 26695 heartbeater.cc:347] Connected to a master server at 127.9.33.126:33077
I20260928 14:46:44.994942 26695 heartbeater.cc:464] Registering TS with master...
I20260928 14:46:44.995200 26695 heartbeater.cc:511] Master 127.9.33.126:33077 requested a full tablet report, sending...
I20260928 14:46:44.995893 26370 ts_manager.cc:194] Registered new tserver with Master: adbfb2890c9945538f3ff3b640cff210 (127.9.33.66:34857)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.67@KRBTEST.COM" created.
I20260928 14:46:44.996496 26370 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.66@KRBTEST.COM'} at 127.9.33.66:39809
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab.
Entry for principal kudu/127.9.33.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab.
I20260928 14:46:45.014432 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.67:0
--local_ip_for_outbound_sockets=127.9.33.67
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:33077
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:37831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:46:45.098201 26700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:46:45.098448 26700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:46:45.098490 26700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:46:45.099774 26700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:46:45.099848 26700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:46:45.099896 26700 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.67
I20260928 14:46:45.101437 26700 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:37831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:33077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26700
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.67
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:46:45.101723 26700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:46:45.101970 26700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:46:45.107196 26705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:45.107188 26708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:45.107191 26706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:45.107432 26700 server_base.cc:1034] running on GCE node
I20260928 14:46:45.107662 26700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:46:45.107950 26700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:46:45.109125 26700 hybrid_clock.cc:648] HybridClock initialized: now 1790606805109122 us; error 37 us; skew 500 ppm
Sep 28 14:46:45 dist-test-slave-pv0w krb5kdc[26325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606805, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:46:45.111644 26700 init.cc:377] Logged in from keytab as kudu/127.9.33.67@KRBTEST.COM (short username kudu)
I20260928 14:46:45.112145 26700 webserver.cc:460] Webserver started at http://127.9.33.67:46699/ using document root <none> and password file <none>
I20260928 14:46:45.112383 26700 fs_manager.cc:359] Metadata directory not provided
I20260928 14:46:45.112457 26700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:46:45.112622 26700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:46:45.113710 26700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-2/data/instance:
uuid: "c2be878186f64bd59296af0827720205"
format_stamp: "Formatted at 2026-09-28 14:46:45 on dist-test-slave-pv0w"
server_key: "460afd174cb0fc53a1d98d2a045d13ac"
server_key_iv: "b9081fbfc72c7812a9ef14d7b9abc5a7"
server_key_version: "encryptionkey@0"
I20260928 14:46:45.114089 26700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance:
uuid: "c2be878186f64bd59296af0827720205"
format_stamp: "Formatted at 2026-09-28 14:46:45 on dist-test-slave-pv0w"
server_key: "460afd174cb0fc53a1d98d2a045d13ac"
server_key_iv: "b9081fbfc72c7812a9ef14d7b9abc5a7"
server_key_version: "encryptionkey@0"
I20260928 14:46:45.115461 26700 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:46:45.116299 26715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:45.116510 26700 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:45.116603 26700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-2/wal
uuid: "c2be878186f64bd59296af0827720205"
format_stamp: "Formatted at 2026-09-28 14:46:45 on dist-test-slave-pv0w"
server_key: "460afd174cb0fc53a1d98d2a045d13ac"
server_key_iv: "b9081fbfc72c7812a9ef14d7b9abc5a7"
server_key_version: "encryptionkey@0"
I20260928 14:46:45.116662 26700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:46:45.128381 26700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:46:45.131264 26700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:46:45.131430 26700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:46:45.131757 26700 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:46:45.132089 26700 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:46:45.132153 26700 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:45.132197 26700 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:46:45.132225 26700 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:45.138358 26700 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.67:44389
I20260928 14:46:45.138401 26828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.67:44389 every 8 connection(s)
I20260928 14:46:45.138773 26700 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
I20260928 14:46:45.139828 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 26700
I20260928 14:46:45.139940 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance
I20260928 14:46:45.140168 9349 external_mini_cluster.cc:1428] Setting key 6c20d73d669ad6798bf3a7002e773986
Sep 28 14:46:45 dist-test-slave-pv0w krb5kdc[26325](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606805, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:46:45.149010 26409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:45.139651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:48905 (local address 127.9.33.126:33077)
0928 14:46:45.139745 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:45.139747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:45.139975 (+ 228us) server_negotiation.cc:407] Connection header received
0928 14:46:45.140708 (+ 733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:45.140709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:45.140720 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:45.140782 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:45.141859 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.142362 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.143453 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.143656 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.143773 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:45.146111 (+ 2338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:45.146114 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:45.146114 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:46:45.146124 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:45.147649 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:45.148126 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:45.148127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:45.148128 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:45.148180 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:45.148413 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:45.148414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:45.148414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:45.148525 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:45.148574 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:45.148862 (+ 288us) server_negotiation.cc:299] Negotiation successful
0928 14:46:45.148940 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 14:46:45.149361 26831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:45.139651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:33077 (local address 127.9.33.67:48905)
0928 14:46:45.139848 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:45.139893 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:45.140603 (+ 710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:45.140857 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:45.140867 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:45.141556 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:46:45.141763 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:45.141768 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.142512 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.142513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:45.143371 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:45.143373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.143823 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.143824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:45.143955 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:45.144459 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:46:45.144468 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:46:45.146008 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:46:45.147798 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:46:45.147812 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:46:45.147814 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:46:45.148055 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:46:45.148243 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:46:45.148243 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:46:45.148244 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:46:45.148353 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:46:45.148596 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:46:45.148602 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:46:45.148738 (+ 136us) client_negotiation.cc:769] Sending connection context
0928 14:46:45.148823 (+ 85us) client_negotiation.cc:240] Negotiation successful
0928 14:46:45.148960 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":124,"threads_started":1}
I20260928 14:46:45.149967 26829 heartbeater.cc:347] Connected to a master server at 127.9.33.126:33077
I20260928 14:46:45.150121 26829 heartbeater.cc:464] Registering TS with master...
I20260928 14:46:45.150350 26829 heartbeater.cc:511] Master 127.9.33.126:33077 requested a full tablet report, sending...
I20260928 14:46:45.150799 26370 ts_manager.cc:194] Registered new tserver with Master: c2be878186f64bd59296af0827720205 (127.9.33.67:44389)
I20260928 14:46:45.151199 26370 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.67@KRBTEST.COM'} at 127.9.33.67:48905
I20260928 14:46:45.152148 9349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:46:45.160178 26409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:45.153682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57248 (local address 127.9.33.126:33077)
0928 14:46:45.153725 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:45.153727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:45.153758 (+ 31us) server_negotiation.cc:407] Connection header received
0928 14:46:45.153818 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:45.153818 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:45.153827 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:45.153886 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:45.154467 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.155027 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.155929 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.156127 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.156186 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:45.157532 (+ 1346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:45.157534 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:45.157534 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:46:45.157542 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:45.159060 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:45.159397 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:45.159398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:45.159398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:45.159449 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:45.159627 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:45.159627 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:45.159627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:45.159846 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:45.159931 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:45.160036 (+ 105us) server_negotiation.cc:299] Negotiation successful
0928 14:46:45.160106 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 14:46:45.162154 26370 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57248:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:46:45.163897 26370 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:46:45.170989 26839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:45.166286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:34857 (local address 127.0.0.1:43040)
0928 14:46:45.167215 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:45.167225 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:45.167272 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:45.167542 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:45.167543 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:45.167547 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:46:45.167585 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:45.167586 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.168535 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.168536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:45.169703 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:45.169706 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.170699 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.170700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:45.170770 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:45.170775 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:46:45.170834 (+ 59us) client_negotiation.cc:240] Negotiation successful
0928 14:46:45.170887 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":916,"mutex_wait_us":68,"thread_start_us":57,"threads_started":1}
I20260928 14:46:45.171032 26840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:45.166453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56340 (local address 127.9.33.67:44389)
0928 14:46:45.166665 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:45.166670 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:45.166684 (+ 14us) server_negotiation.cc:407] Connection header received
0928 14:46:45.166817 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:45.166819 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:45.166832 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:45.166909 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:46:45.167196 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.168405 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.169995 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.170707 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.170773 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:45.170824 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:45.170898 (+ 74us) server_negotiation.cc:299] Negotiation successful
0928 14:46:45.170938 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":92,"threads_started":1}
I20260928 14:46:45.170990 26838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:45.166274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:44171 (local address 127.0.0.1:52800)
0928 14:46:45.166725 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:45.166734 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:45.166775 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:45.167122 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:45.167124 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:45.167126 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:46:45.167178 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:45.167179 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.168543 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.168544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:45.169906 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:45.169909 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.170622 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.170623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:45.170713 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:45.170727 (+ 14us) client_negotiation.cc:769] Sending connection context
0928 14:46:45.170799 (+ 72us) client_negotiation.cc:240] Negotiation successful
0928 14:46:45.170865 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"mutex_wait_us":106,"thread_start_us":60,"threads_started":1}
I20260928 14:46:45.171329 26842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:45.166548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43040 (local address 127.9.33.66:34857)
0928 14:46:45.166698 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:45.166703 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:45.167331 (+ 628us) server_negotiation.cc:407] Connection header received
0928 14:46:45.167376 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:45.167378 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:45.167395 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:45.167466 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:46:45.167668 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.168412 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.169790 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.170523 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.170596 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:45.170660 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:45.171225 (+ 565us) server_negotiation.cc:299] Negotiation successful
0928 14:46:45.171265 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":79,"threads_started":1}
I20260928 14:46:45.171373 26841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:45.166460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52800 (local address 127.9.33.65:44171)
0928 14:46:45.166924 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:45.166928 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:45.166936 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:46:45.166974 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:45.166976 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:45.166994 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:45.167094 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:46:45.167275 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.168405 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.170012 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.170516 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.170613 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:45.170673 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:45.171031 (+ 358us) server_negotiation.cc:299] Negotiation successful
0928 14:46:45.171291 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":91,"threads_started":1}
I20260928 14:46:45.171001 26837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:45.166271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:44389 (local address 127.0.0.1:56340)
0928 14:46:45.166590 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:45.166637 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:45.166710 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:45.166932 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:45.166936 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:45.166938 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:46:45.167026 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:45.167029 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.168527 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.168528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:45.169905 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:45.169909 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.170775 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.170776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:45.170833 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:45.170836 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:46:45.170885 (+ 49us) client_negotiation.cc:240] Negotiation successful
0928 14:46:45.170923 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":102,"thread_start_us":67,"threads_started":1}
I20260928 14:46:45.172175 26495 tablet_service.cc:1467] Processing CreateTablet for tablet 9cefd2100c9147d284fb5d9bf7c00b96 (DEFAULT_TABLE table=test-table [id=ed1bcf14f0674b808770b6d948217906]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:46:45.172348 26763 tablet_service.cc:1467] Processing CreateTablet for tablet 9cefd2100c9147d284fb5d9bf7c00b96 (DEFAULT_TABLE table=test-table [id=ed1bcf14f0674b808770b6d948217906]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:46:45.172569 26495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cefd2100c9147d284fb5d9bf7c00b96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:46:45.172719 26763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cefd2100c9147d284fb5d9bf7c00b96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:46:45.175570 26843 tablet_bootstrap.cc:492] T 9cefd2100c9147d284fb5d9bf7c00b96 P 84b32c8c7b5042b3a4f24fd4b6e01af6: Bootstrap starting.
I20260928 14:46:45.176038 26629 tablet_service.cc:1467] Processing CreateTablet for tablet 9cefd2100c9147d284fb5d9bf7c00b96 (DEFAULT_TABLE table=test-table [id=ed1bcf14f0674b808770b6d948217906]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:46:45.176360 26629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cefd2100c9147d284fb5d9bf7c00b96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:46:45.176414 26843 tablet_bootstrap.cc:654] T 9cefd2100c9147d284fb5d9bf7c00b96 P 84b32c8c7b5042b3a4f24fd4b6e01af6: Neither blocks nor log segments found. Creating new log.
I20260928 14:46:45.176947 26843 log.cc:824] T 9cefd2100c9147d284fb5d9bf7c00b96 P 84b32c8c7b5042b3a4f24fd4b6e01af6: Log is configured to *not* fsync() on all Append() calls
I20260928 14:46:45.177290 26844 tablet_bootstrap.cc:492] T 9cefd2100c9147d284fb5d9bf7c00b96 P c2be878186f64bd59296af0827720205: Bootstrap starting.
I20260928 14:46:45.177764 26843 tablet_bootstrap.cc:492] T 9cefd2100c9147d284fb5d9bf7c00b96 P 84b32c8c7b5042b3a4f24fd4b6e01af6: No bootstrap required, opened a new log
I20260928 14:46:45.177848 26843 ts_tablet_manager.cc:1397] T 9cefd2100c9147d284fb5d9bf7c00b96 P 84b32c8c7b5042b3a4f24fd4b6e01af6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:46:45.177917 26844 tablet_bootstrap.cc:654] T 9cefd2100c9147d284fb5d9bf7c00b96 P c2be878186f64bd59296af0827720205: Neither blocks nor log segments found. Creating new log.
I20260928 14:46:45.178435 26844 log.cc:824] T 9cefd2100c9147d284fb5d9bf7c00b96 P c2be878186f64bd59296af0827720205: Log is configured to *not* fsync() on all Append() calls
I20260928 14:46:45.179123 26844 tablet_bootstrap.cc:492] T 9cefd2100c9147d284fb5d9bf7c00b96 P c2be878186f64bd59296af0827720205: No bootstrap required, opened a new log
I20260928 14:46:45.179214 26844 ts_tablet_manager.cc:1397] T 9cefd2100c9147d284fb5d9bf7c00b96 P c2be878186f64bd59296af0827720205: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:46:45.179773 26847 tablet_bootstrap.cc:492] T 9cefd2100c9147d284fb5d9bf7c00b96 P adbfb2890c9945538f3ff3b640cff210: Bootstrap starting.
I20260928 14:46:45.180197 26843 raft_consensus.cc:348] T 9cefd2100c9147d284fb5d9bf7c00b96 P 84b32c8c7b5042b3a4f24fd4b6e01af6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be878186f64bd59296af0827720205" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 44389 } } peers { permanent_uuid: "84b32c8c7b5042b3a4f24fd4b6e01af6" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 44171 } } peers { permanent_uuid: "adbfb2890c9945538f3ff3b640cff210" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 34857 } }
I20260928 14:46:45.180377 26843 raft_consensus.cc:374] T 9cefd2100c9147d284fb5d9bf7c00b96 P 84b32c8c7b5042b3a4f24fd4b6e01af6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:46:45.180421 26843 raft_consensus.cc:729] T 9cefd2100c9147d284fb5d9bf7c00b96 P 84b32c8c7b5042b3a4f24fd4b6e01af6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84b32c8c7b5042b3a4f24fd4b6e01af6, State: Initialized, Role: FOLLOWER
I20260928 14:46:45.180490 26847 tablet_bootstrap.cc:654] T 9cefd2100c9147d284fb5d9bf7c00b96 P adbfb2890c9945538f3ff3b640cff210: Neither blocks nor log segments found. Creating new log.
I20260928 14:46:45.180583 26843 consensus_queue.cc:260] T 9cefd2100c9147d284fb5d9bf7c00b96 P 84b32c8c7b5042b3a4f24fd4b6e01af6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be878186f64bd59296af0827720205" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 44389 } } peers { permanent_uuid: "84b32c8c7b5042b3a4f24fd4b6e01af6" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 44171 } } peers { permanent_uuid: "adbfb2890c9945538f3ff3b640cff210" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 34857 } }
I20260928 14:46:45.180909 26561 heartbeater.cc:503] Master 127.9.33.126:33077 was elected leader, sending a full tablet report...
I20260928 14:46:45.181102 26847 log.cc:824] T 9cefd2100c9147d284fb5d9bf7c00b96 P adbfb2890c9945538f3ff3b640cff210: Log is configured to *not* fsync() on all Append() calls
I20260928 14:46:45.180909 26843 ts_tablet_manager.cc:1428] T 9cefd2100c9147d284fb5d9bf7c00b96 P 84b32c8c7b5042b3a4f24fd4b6e01af6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:46:45.181401 26844 raft_consensus.cc:348] T 9cefd2100c9147d284fb5d9bf7c00b96 P c2be878186f64bd59296af0827720205 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be878186f64bd59296af0827720205" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 44389 } } peers { permanent_uuid: "84b32c8c7b5042b3a4f24fd4b6e01af6" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 44171 } } peers { permanent_uuid: "adbfb2890c9945538f3ff3b640cff210" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 34857 } }
I20260928 14:46:45.181581 26844 raft_consensus.cc:374] T 9cefd2100c9147d284fb5d9bf7c00b96 P c2be878186f64bd59296af0827720205 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:46:45.181638 26844 raft_consensus.cc:729] T 9cefd2100c9147d284fb5d9bf7c00b96 P c2be878186f64bd59296af0827720205 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2be878186f64bd59296af0827720205, State: Initialized, Role: FOLLOWER
I20260928 14:46:45.181757 26844 consensus_queue.cc:260] T 9cefd2100c9147d284fb5d9bf7c00b96 P c2be878186f64bd59296af0827720205 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be878186f64bd59296af0827720205" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 44389 } } peers { permanent_uuid: "84b32c8c7b5042b3a4f24fd4b6e01af6" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 44171 } } peers { permanent_uuid: "adbfb2890c9945538f3ff3b640cff210" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 34857 } }
I20260928 14:46:45.181895 26847 tablet_bootstrap.cc:492] T 9cefd2100c9147d284fb5d9bf7c00b96 P adbfb2890c9945538f3ff3b640cff210: No bootstrap required, opened a new log
I20260928 14:46:45.181957 26847 ts_tablet_manager.cc:1397] T 9cefd2100c9147d284fb5d9bf7c00b96 P adbfb2890c9945538f3ff3b640cff210: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:46:45.182054 26844 ts_tablet_manager.cc:1428] T 9cefd2100c9147d284fb5d9bf7c00b96 P c2be878186f64bd59296af0827720205: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:46:45.182098 26829 heartbeater.cc:503] Master 127.9.33.126:33077 was elected leader, sending a full tablet report...
I20260928 14:46:45.183796 26847 raft_consensus.cc:348] T 9cefd2100c9147d284fb5d9bf7c00b96 P adbfb2890c9945538f3ff3b640cff210 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be878186f64bd59296af0827720205" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 44389 } } peers { permanent_uuid: "84b32c8c7b5042b3a4f24fd4b6e01af6" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 44171 } } peers { permanent_uuid: "adbfb2890c9945538f3ff3b640cff210" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 34857 } }
I20260928 14:46:45.183988 26847 raft_consensus.cc:374] T 9cefd2100c9147d284fb5d9bf7c00b96 P adbfb2890c9945538f3ff3b640cff210 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:46:45.184044 26847 raft_consensus.cc:729] T 9cefd2100c9147d284fb5d9bf7c00b96 P adbfb2890c9945538f3ff3b640cff210 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adbfb2890c9945538f3ff3b640cff210, State: Initialized, Role: FOLLOWER
I20260928 14:46:45.184163 26847 consensus_queue.cc:260] T 9cefd2100c9147d284fb5d9bf7c00b96 P adbfb2890c9945538f3ff3b640cff210 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be878186f64bd59296af0827720205" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 44389 } } peers { permanent_uuid: "84b32c8c7b5042b3a4f24fd4b6e01af6" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 44171 } } peers { permanent_uuid: "adbfb2890c9945538f3ff3b640cff210" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 34857 } }
I20260928 14:46:45.184418 26847 ts_tablet_manager.cc:1428] T 9cefd2100c9147d284fb5d9bf7c00b96 P adbfb2890c9945538f3ff3b640cff210: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260928 14:46:45.184484 26695 heartbeater.cc:503] Master 127.9.33.126:33077 was elected leader, sending a full tablet report...
I20260928 14:46:45.185328 26851 raft_consensus.cc:482] T 9cefd2100c9147d284fb5d9bf7c00b96 P adbfb2890c9945538f3ff3b640cff210 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:46:45.185390 26851 raft_consensus.cc:504] T 9cefd2100c9147d284fb5d9bf7c00b96 P adbfb2890c9945538f3ff3b640cff210 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be878186f64bd59296af0827720205" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 44389 } } peers { permanent_uuid: "84b32c8c7b5042b3a4f24fd4b6e01af6" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 44171 } } peers { permanent_uuid: "adbfb2890c9945538f3ff3b640cff210" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 34857 } }
I20260928 14:46:45.185674 26851 leader_election.cc:288] T 9cefd2100c9147d284fb5d9bf7c00b96 P adbfb2890c9945538f3ff3b640cff210 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2be878186f64bd59296af0827720205 (127.9.33.67:44389), 84b32c8c7b5042b3a4f24fd4b6e01af6 (127.9.33.65:44171)
I20260928 14:46:45.188844 26840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:45.185917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:44281 (local address 127.9.33.67:44389)
0928 14:46:45.185954 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:45.185956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:45.185964 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:46:45.185987 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:45.185988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:45.185998 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:45.186095 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:46:45.186248 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.186919 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.188018 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.188494 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.188549 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:45.188559 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:45.188728 (+ 169us) server_negotiation.cc:299] Negotiation successful
0928 14:46:45.188766 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260928 14:46:45.188864 26697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:45.185798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:44389 (local address 127.9.33.66:44281)
0928 14:46:45.185842 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:45.185855 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:45.185897 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:45.186100 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:45.186101 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:45.186103 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:46:45.186168 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:45.186171 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.187010 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.187011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:45.187898 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:45.187901 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.188573 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.188574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:45.188657 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:45.188662 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:46:45.188717 (+ 55us) client_negotiation.cc:240] Negotiation successful
0928 14:46:45.188765 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20,"mutex_wait_us":1}
I20260928 14:46:45.189028 26841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:45.185864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:51759 (local address 127.9.33.65:44171)
0928 14:46:45.185907 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:45.185909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:45.186114 (+ 205us) server_negotiation.cc:407] Connection header received
0928 14:46:45.186158 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:45.186159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:45.186170 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:45.186215 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:46:45.186364 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.186833 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.188209 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.188651 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.188725 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:45.188735 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:45.188907 (+ 172us) server_negotiation.cc:299] Negotiation successful
0928 14:46:45.188957 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 14:46:45.189184 26783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cefd2100c9147d284fb5d9bf7c00b96" candidate_uuid: "adbfb2890c9945538f3ff3b640cff210" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be878186f64bd59296af0827720205" is_pre_election: true
I20260928 14:46:45.189255 26852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:45.185805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:44171 (local address 127.9.33.66:51759)
0928 14:46:45.186038 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:45.186046 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:45.186094 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:45.186216 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:45.186217 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:45.186218 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:46:45.186284 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:45.186286 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.186917 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.186918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:45.188108 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:45.188111 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.188737 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.188738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:45.188806 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:45.188811 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:46:45.189112 (+ 301us) client_negotiation.cc:240] Negotiation successful
0928 14:46:45.189169 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":133,"thread_start_us":78,"threads_started":1}
I20260928 14:46:45.189378 26783 raft_consensus.cc:2454] T 9cefd2100c9147d284fb5d9bf7c00b96 P c2be878186f64bd59296af0827720205 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate adbfb2890c9945538f3ff3b640cff210 in term 0.
I20260928 14:46:45.189633 26583 leader_election.cc:302] T 9cefd2100c9147d284fb5d9bf7c00b96 P adbfb2890c9945538f3ff3b640cff210 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: adbfb2890c9945538f3ff3b640cff210, c2be878186f64bd59296af0827720205; no voters:
I20260928 14:46:45.189702 26515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cefd2100c9147d284fb5d9bf7c00b96" candidate_uuid: "adbfb2890c9945538f3ff3b640cff210" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b32c8c7b5042b3a4f24fd4b6e01af6" is_pre_election: true
I20260928 14:46:45.189774 26851 raft_consensus.cc:2781] T 9cefd2100c9147d284fb5d9bf7c00b96 P adbfb2890c9945538f3ff3b640cff210 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:46:45.189842 26851 raft_consensus.cc:482] T 9cefd2100c9147d284fb5d9bf7c00b96 P adbfb2890c9945538f3ff3b640cff210 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:46:45.189864 26515 raft_consensus.cc:2454] T 9cefd2100c9147d284fb5d9bf7c00b96 P 84b32c8c7b5042b3a4f24fd4b6e01af6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate adbfb2890c9945538f3ff3b640cff210 in term 0.
I20260928 14:46:45.189877 26851 raft_consensus.cc:3037] T 9cefd2100c9147d284fb5d9bf7c00b96 P adbfb2890c9945538f3ff3b640cff210 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:46:45.190634 26851 raft_consensus.cc:504] T 9cefd2100c9147d284fb5d9bf7c00b96 P adbfb2890c9945538f3ff3b640cff210 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be878186f64bd59296af0827720205" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 44389 } } peers { permanent_uuid: "84b32c8c7b5042b3a4f24fd4b6e01af6" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 44171 } } peers { permanent_uuid: "adbfb2890c9945538f3ff3b640cff210" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 34857 } }
I20260928 14:46:45.190794 26851 leader_election.cc:288] T 9cefd2100c9147d284fb5d9bf7c00b96 P adbfb2890c9945538f3ff3b640cff210 [CANDIDATE]: Term 1 election: Requested vote from peers c2be878186f64bd59296af0827720205 (127.9.33.67:44389), 84b32c8c7b5042b3a4f24fd4b6e01af6 (127.9.33.65:44171)
I20260928 14:46:45.190943 26515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cefd2100c9147d284fb5d9bf7c00b96" candidate_uuid: "adbfb2890c9945538f3ff3b640cff210" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84b32c8c7b5042b3a4f24fd4b6e01af6"
I20260928 14:46:45.190992 26783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cefd2100c9147d284fb5d9bf7c00b96" candidate_uuid: "adbfb2890c9945538f3ff3b640cff210" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2be878186f64bd59296af0827720205"
I20260928 14:46:45.191041 26515 raft_consensus.cc:3037] T 9cefd2100c9147d284fb5d9bf7c00b96 P 84b32c8c7b5042b3a4f24fd4b6e01af6 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:46:45.191083 26783 raft_consensus.cc:3037] T 9cefd2100c9147d284fb5d9bf7c00b96 P c2be878186f64bd59296af0827720205 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:46:45.191860 26783 raft_consensus.cc:2454] T 9cefd2100c9147d284fb5d9bf7c00b96 P c2be878186f64bd59296af0827720205 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate adbfb2890c9945538f3ff3b640cff210 in term 1.
I20260928 14:46:45.191911 26515 raft_consensus.cc:2454] T 9cefd2100c9147d284fb5d9bf7c00b96 P 84b32c8c7b5042b3a4f24fd4b6e01af6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate adbfb2890c9945538f3ff3b640cff210 in term 1.
I20260928 14:46:45.192056 26583 leader_election.cc:302] T 9cefd2100c9147d284fb5d9bf7c00b96 P adbfb2890c9945538f3ff3b640cff210 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: adbfb2890c9945538f3ff3b640cff210, c2be878186f64bd59296af0827720205; no voters:
I20260928 14:46:45.192184 26851 raft_consensus.cc:2781] T 9cefd2100c9147d284fb5d9bf7c00b96 P adbfb2890c9945538f3ff3b640cff210 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:46:45.192360 26851 raft_consensus.cc:686] T 9cefd2100c9147d284fb5d9bf7c00b96 P adbfb2890c9945538f3ff3b640cff210 [term 1 LEADER]: Becoming Leader. State: Replica: adbfb2890c9945538f3ff3b640cff210, State: Running, Role: LEADER
I20260928 14:46:45.192471 26851 consensus_queue.cc:237] T 9cefd2100c9147d284fb5d9bf7c00b96 P adbfb2890c9945538f3ff3b640cff210 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be878186f64bd59296af0827720205" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 44389 } } peers { permanent_uuid: "84b32c8c7b5042b3a4f24fd4b6e01af6" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 44171 } } peers { permanent_uuid: "adbfb2890c9945538f3ff3b640cff210" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 34857 } }
I20260928 14:46:45.193346 26370 catalog_manager.cc:5495] T 9cefd2100c9147d284fb5d9bf7c00b96 P adbfb2890c9945538f3ff3b640cff210 reported cstate change: term changed from 0 to 1, leader changed from <none> to adbfb2890c9945538f3ff3b640cff210 (127.9.33.66). New cstate: current_term: 1 leader_uuid: "adbfb2890c9945538f3ff3b640cff210" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c2be878186f64bd59296af0827720205" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 44389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84b32c8c7b5042b3a4f24fd4b6e01af6" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 44171 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "adbfb2890c9945538f3ff3b640cff210" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 34857 } health_report { overall_health: HEALTHY } } }
I20260928 14:46:45.199227 26842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:45.196743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43042 (local address 127.9.33.66:34857)
0928 14:46:45.196789 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:45.196792 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:45.196810 (+ 18us) server_negotiation.cc:407] Connection header received
0928 14:46:45.196844 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:45.196845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:45.196874 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:45.196981 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:46:45.197102 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.197573 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.198354 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.198573 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.198653 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:45.198845 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:46:45.198965 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:46:45.199066 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:45.199092 (+ 26us) server_negotiation.cc:299] Negotiation successful
0928 14:46:45.199115 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 14:46:45.201162 26783 raft_consensus.cc:1261] T 9cefd2100c9147d284fb5d9bf7c00b96 P c2be878186f64bd59296af0827720205 [term 1 FOLLOWER]: Refusing update from remote peer adbfb2890c9945538f3ff3b640cff210: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:46:45.201192 26515 raft_consensus.cc:1261] T 9cefd2100c9147d284fb5d9bf7c00b96 P 84b32c8c7b5042b3a4f24fd4b6e01af6 [term 1 FOLLOWER]: Refusing update from remote peer adbfb2890c9945538f3ff3b640cff210: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:46:45.201596 26853 consensus_queue.cc:1035] T 9cefd2100c9147d284fb5d9bf7c00b96 P adbfb2890c9945538f3ff3b640cff210 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84b32c8c7b5042b3a4f24fd4b6e01af6" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 44171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:46:45.201731 26851 consensus_queue.cc:1035] T 9cefd2100c9147d284fb5d9bf7c00b96 P adbfb2890c9945538f3ff3b640cff210 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2be878186f64bd59296af0827720205" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 44389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:46:45.204149 26856 mvcc.cc:205] Tried to move back new op lower bound from 7334325474101202944 to 7334325474069057536. Current Snapshot: MvccSnapshot[applied={T|T < 7334325474101202944}]
I20260928 14:46:45.204367 26858 mvcc.cc:205] Tried to move back new op lower bound from 7334325474101202944 to 7334325474069057536. Current Snapshot: MvccSnapshot[applied={T|T < 7334325474101202944}]
I20260928 14:46:45.204763 26857 mvcc.cc:205] Tried to move back new op lower bound from 7334325474101202944 to 7334325474069057536. Current Snapshot: MvccSnapshot[applied={T|T < 7334325474101202944}]
I20260928 14:46:45.207540 26370 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:46:45.207659 26370 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:46:45.208658 26370 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d2b8b32cd503484ba11cba08a8ec588e: Sending DeleteTablet for 3 replicas of tablet 9cefd2100c9147d284fb5d9bf7c00b96
I20260928 14:46:45.209002 26763 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cefd2100c9147d284fb5d9bf7c00b96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:46:45 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:56340
I20260928 14:46:45.209039 26629 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cefd2100c9147d284fb5d9bf7c00b96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:46:45 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:43040
I20260928 14:46:45.209142 26495 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cefd2100c9147d284fb5d9bf7c00b96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:46:45 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:52800
I20260928 14:46:45.209347 26867 tablet_replica.cc:330] T 9cefd2100c9147d284fb5d9bf7c00b96 P 84b32c8c7b5042b3a4f24fd4b6e01af6: stopping tablet replica
I20260928 14:46:45.209481 26867 raft_consensus.cc:2229] T 9cefd2100c9147d284fb5d9bf7c00b96 P 84b32c8c7b5042b3a4f24fd4b6e01af6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:46:45.209528 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 26432
I20260928 14:46:45.209597 26867 raft_consensus.cc:2258] T 9cefd2100c9147d284fb5d9bf7c00b96 P 84b32c8c7b5042b3a4f24fd4b6e01af6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:46:45.209764 26866 tablet_replica.cc:330] T 9cefd2100c9147d284fb5d9bf7c00b96 P adbfb2890c9945538f3ff3b640cff210: stopping tablet replica
I20260928 14:46:45.209856 26866 raft_consensus.cc:2229] T 9cefd2100c9147d284fb5d9bf7c00b96 P adbfb2890c9945538f3ff3b640cff210 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:46:45.210371 26866 raft_consensus.cc:2258] T 9cefd2100c9147d284fb5d9bf7c00b96 P adbfb2890c9945538f3ff3b640cff210 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:46:45.210855 26866 ts_tablet_manager.cc:1905] T 9cefd2100c9147d284fb5d9bf7c00b96 P adbfb2890c9945538f3ff3b640cff210: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:46:45.212541 26866 ts_tablet_manager.cc:1918] T 9cefd2100c9147d284fb5d9bf7c00b96 P adbfb2890c9945538f3ff3b640cff210: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:46:45.212620 26866 log.cc:1196] T 9cefd2100c9147d284fb5d9bf7c00b96 P adbfb2890c9945538f3ff3b640cff210: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-1/wal/wals/9cefd2100c9147d284fb5d9bf7c00b96
I20260928 14:46:45.212901 26866 ts_tablet_manager.cc:1939] T 9cefd2100c9147d284fb5d9bf7c00b96 P adbfb2890c9945538f3ff3b640cff210: Deleting consensus metadata
I20260928 14:46:45.213438 26357 catalog_manager.cc:4841] TS adbfb2890c9945538f3ff3b640cff210 (127.9.33.66:34857): tablet 9cefd2100c9147d284fb5d9bf7c00b96 (table test-table [id=ed1bcf14f0674b808770b6d948217906]) successfully deleted
W20260928 14:46:45.215811 26356 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.33.65:44171 (error 108)
W20260928 14:46:45.216377 26837 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:46:45.216235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:44171 (local address 127.0.0.1:52810)
0928 14:46:45.216291 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:45.216327 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.65:44171: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":35}
W20260928 14:46:45.216502 26356 catalog_manager.cc:4584] TS 84b32c8c7b5042b3a4f24fd4b6e01af6 (127.9.33.65:44171): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9cefd2100c9147d284fb5d9bf7c00b96: Network error: Client connection negotiation failed: client connection to 127.9.33.65:44171: connect: Connection reset by peer (error 104)
I20260928 14:46:45.216621 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 26566
I20260928 14:46:45.217825 26865 tablet_replica.cc:330] T 9cefd2100c9147d284fb5d9bf7c00b96 P c2be878186f64bd59296af0827720205: stopping tablet replica
I20260928 14:46:45.217993 26865 raft_consensus.cc:2229] T 9cefd2100c9147d284fb5d9bf7c00b96 P c2be878186f64bd59296af0827720205 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:46:45.218153 26865 raft_consensus.cc:2258] T 9cefd2100c9147d284fb5d9bf7c00b96 P c2be878186f64bd59296af0827720205 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:46:45.218719 26865 ts_tablet_manager.cc:1905] T 9cefd2100c9147d284fb5d9bf7c00b96 P c2be878186f64bd59296af0827720205: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:46:45.220964 26865 ts_tablet_manager.cc:1918] T 9cefd2100c9147d284fb5d9bf7c00b96 P c2be878186f64bd59296af0827720205: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:46:45.221058 26865 log.cc:1196] T 9cefd2100c9147d284fb5d9bf7c00b96 P c2be878186f64bd59296af0827720205: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790606561545179-9349-0/minicluster-data/ts-2/wal/wals/9cefd2100c9147d284fb5d9bf7c00b96
I20260928 14:46:45.221364 26865 ts_tablet_manager.cc:1939] T 9cefd2100c9147d284fb5d9bf7c00b96 P c2be878186f64bd59296af0827720205: Deleting consensus metadata
I20260928 14:46:45.222084 26355 catalog_manager.cc:4841] TS c2be878186f64bd59296af0827720205 (127.9.33.67:44389): tablet 9cefd2100c9147d284fb5d9bf7c00b96 (table test-table [id=ed1bcf14f0674b808770b6d948217906]) successfully deleted
I20260928 14:46:45.223091 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 26700
I20260928 14:46:45.228132 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 26338
2026-09-28T14:46:45Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1944 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-28T14:46:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:46:45Z Disabled control of system clock
I20260928 14:46:45.257272 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:39343
--webserver_interface=127.9.33.126
--webserver_port=0
--builtin_ntp_servers=127.9.33.84:35779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:39343
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 14:46:45.343329 26872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:46:45.343593 26872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:46:45.343617 26872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:46:45.344985 26872 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:46:45.345043 26872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:46:45.345057 26872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:46:45.345070 26872 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:46:45.345083 26872 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:46:45.346580 26872 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:35779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:39343
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:39343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:46:45.346882 26872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:46:45.347206 26872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:46:45.353096 26878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:45.353088 26877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:45.353295 26872 server_base.cc:1034] running on GCE node
W20260928 14:46:45.353088 26880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:45.353686 26872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:46:45.354039 26872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:46:45.355239 26872 hybrid_clock.cc:648] HybridClock initialized: now 1790606805355220 us; error 48 us; skew 500 ppm
I20260928 14:46:45.356781 26872 webserver.cc:460] Webserver started at http://127.9.33.126:45561/ using document root <none> and password file <none>
I20260928 14:46:45.357053 26872 fs_manager.cc:359] Metadata directory not provided
I20260928 14:46:45.357165 26872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:46:45.357317 26872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:46:45.358474 26872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "69ea972f32794056be7cc25ccac94002"
format_stamp: "Formatted at 2026-09-28 14:46:45 on dist-test-slave-pv0w"
server_key: "6d35e3dee5dfb107e8184832e7af2a56"
server_key_iv: "3cf0c48c40262dd15d59d1b44ce1a21e"
server_key_version: "encryptionkey@0"
I20260928 14:46:45.358871 26872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "69ea972f32794056be7cc25ccac94002"
format_stamp: "Formatted at 2026-09-28 14:46:45 on dist-test-slave-pv0w"
server_key: "6d35e3dee5dfb107e8184832e7af2a56"
server_key_iv: "3cf0c48c40262dd15d59d1b44ce1a21e"
server_key_version: "encryptionkey@0"
I20260928 14:46:45.360424 26872 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:46:45.361294 26886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:45.361557 26872 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:46:45.361660 26872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "69ea972f32794056be7cc25ccac94002"
format_stamp: "Formatted at 2026-09-28 14:46:45 on dist-test-slave-pv0w"
server_key: "6d35e3dee5dfb107e8184832e7af2a56"
server_key_iv: "3cf0c48c40262dd15d59d1b44ce1a21e"
server_key_version: "encryptionkey@0"
I20260928 14:46:45.361757 26872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:46:45.378139 26872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:46:45.378532 26872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:46:45.378719 26872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:46:45.383077 26872 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:39343
I20260928 14:46:45.383122 26938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:39343 every 8 connection(s)
I20260928 14:46:45.383507 26872 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:46:45.384610 26939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:46:45.386976 26939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69ea972f32794056be7cc25ccac94002: Bootstrap starting.
I20260928 14:46:45.387530 26939 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69ea972f32794056be7cc25ccac94002: Neither blocks nor log segments found. Creating new log.
I20260928 14:46:45.387897 26939 log.cc:824] T 00000000000000000000000000000000 P 69ea972f32794056be7cc25ccac94002: Log is configured to *not* fsync() on all Append() calls
I20260928 14:46:45.388657 26939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69ea972f32794056be7cc25ccac94002: No bootstrap required, opened a new log
I20260928 14:46:45.390338 26939 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69ea972f32794056be7cc25ccac94002 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69ea972f32794056be7cc25ccac94002" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 39343 } }
I20260928 14:46:45.390499 26939 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69ea972f32794056be7cc25ccac94002 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:46:45.390534 26939 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69ea972f32794056be7cc25ccac94002 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69ea972f32794056be7cc25ccac94002, State: Initialized, Role: FOLLOWER
I20260928 14:46:45.390708 26939 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69ea972f32794056be7cc25ccac94002 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69ea972f32794056be7cc25ccac94002" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 39343 } }
I20260928 14:46:45.390808 26939 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69ea972f32794056be7cc25ccac94002 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:46:45.390839 26939 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69ea972f32794056be7cc25ccac94002 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:46:45.390877 26939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69ea972f32794056be7cc25ccac94002 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:46:45.391427 26939 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69ea972f32794056be7cc25ccac94002 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69ea972f32794056be7cc25ccac94002" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 39343 } }
I20260928 14:46:45.391528 26939 leader_election.cc:302] T 00000000000000000000000000000000 P 69ea972f32794056be7cc25ccac94002 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69ea972f32794056be7cc25ccac94002; no voters:
I20260928 14:46:45.391688 26939 leader_election.cc:288] T 00000000000000000000000000000000 P 69ea972f32794056be7cc25ccac94002 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:46:45.391798 26942 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69ea972f32794056be7cc25ccac94002 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:46:45.392050 26939 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69ea972f32794056be7cc25ccac94002 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:46:45.392176 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 26872
I20260928 14:46:45.392285 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:46:45.392480 9349 external_mini_cluster.cc:1428] Setting key 471fc9f4cff59b2dc2326218cd85007c
I20260928 14:46:45.392740 26942 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69ea972f32794056be7cc25ccac94002 [term 1 LEADER]: Becoming Leader. State: Replica: 69ea972f32794056be7cc25ccac94002, State: Running, Role: LEADER
I20260928 14:46:45.392871 26942 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69ea972f32794056be7cc25ccac94002 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69ea972f32794056be7cc25ccac94002" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 39343 } }
I20260928 14:46:45.393481 26943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69ea972f32794056be7cc25ccac94002 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69ea972f32794056be7cc25ccac94002" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69ea972f32794056be7cc25ccac94002" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 39343 } } }
I20260928 14:46:45.393761 26943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69ea972f32794056be7cc25ccac94002 [sys.catalog]: This master's current role is: LEADER
I20260928 14:46:45.393481 26944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69ea972f32794056be7cc25ccac94002 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69ea972f32794056be7cc25ccac94002. Latest consensus state: current_term: 1 leader_uuid: "69ea972f32794056be7cc25ccac94002" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69ea972f32794056be7cc25ccac94002" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 39343 } } }
I20260928 14:46:45.393929 26944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69ea972f32794056be7cc25ccac94002 [sys.catalog]: This master's current role is: LEADER
W20260928 14:46:45.394048 26958 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 69ea972f32794056be7cc25ccac94002: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:46:45.394100 26958 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:46:45.394322 26960 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:46:45.394512 26960 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:46:45.396157 26960 catalog_manager.cc:1347] Generated new cluster ID: 70710fe656e542da88faf67ad94ce0e9
I20260928 14:46:45.396245 26960 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:46:45.399912 26951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:45.393054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44506 (local address 127.9.33.126:39343)
0928 14:46:45.393303 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:45.393316 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:45.393709 (+ 393us) server_negotiation.cc:407] Connection header received
0928 14:46:45.394437 (+ 728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:45.394445 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:45.394520 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:45.394668 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:45.394921 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.396369 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.397528 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.398212 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.398351 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:45.399075 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:45.399079 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:45.399079 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:46:45.399302 (+ 223us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:45.399350 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:46:45.399366 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:45.399447 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:45.399533 (+ 86us) server_negotiation.cc:299] Negotiation successful
0928 14:46:45.399627 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":93,"threads_started":1}
I20260928 14:46:45.408962 26960 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:46:45.409709 26960 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:46:45.414392 26960 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69ea972f32794056be7cc25ccac94002: Generated new TSK 0
I20260928 14:46:45.414611 26960 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:46:45.420025 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.65:0
--local_ip_for_outbound_sockets=127.9.33.65
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:39343
--builtin_ntp_servers=127.9.33.84:35779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 14:46:45.504590 26963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:46:45.504853 26963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:46:45.504892 26963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:46:45.506143 26963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:46:45.506211 26963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:46:45.506258 26963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.65
I20260928 14:46:45.507696 26963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:35779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:39343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.65
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:46:45.508041 26963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:46:45.508378 26963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:46:45.514546 26969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:45.514669 26971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:45.514555 26968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:45.514846 26963 server_base.cc:1034] running on GCE node
I20260928 14:46:45.515059 26963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:46:45.515353 26963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:46:45.516523 26963 hybrid_clock.cc:648] HybridClock initialized: now 1790606805516509 us; error 43 us; skew 500 ppm
I20260928 14:46:45.518028 26963 webserver.cc:460] Webserver started at http://127.9.33.65:34353/ using document root <none> and password file <none>
I20260928 14:46:45.518316 26963 fs_manager.cc:359] Metadata directory not provided
I20260928 14:46:45.518365 26963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:46:45.518513 26963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:46:45.519636 26963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-0/data/instance:
uuid: "ae5c1445a5404ee2871e96504045bb37"
format_stamp: "Formatted at 2026-09-28 14:46:45 on dist-test-slave-pv0w"
server_key: "92504293524417f43492dd22101fa8be"
server_key_iv: "e560c9830191b67156bdd9d583fbdb4a"
server_key_version: "encryptionkey@0"
I20260928 14:46:45.520076 26963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance:
uuid: "ae5c1445a5404ee2871e96504045bb37"
format_stamp: "Formatted at 2026-09-28 14:46:45 on dist-test-slave-pv0w"
server_key: "92504293524417f43492dd22101fa8be"
server_key_iv: "e560c9830191b67156bdd9d583fbdb4a"
server_key_version: "encryptionkey@0"
I20260928 14:46:45.521473 26963 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:46:45.522280 26977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:45.522435 26963 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:46:45.522547 26963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-0/wal
uuid: "ae5c1445a5404ee2871e96504045bb37"
format_stamp: "Formatted at 2026-09-28 14:46:45 on dist-test-slave-pv0w"
server_key: "92504293524417f43492dd22101fa8be"
server_key_iv: "e560c9830191b67156bdd9d583fbdb4a"
server_key_version: "encryptionkey@0"
I20260928 14:46:45.522631 26963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:46:45.542074 26963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:46:45.542443 26963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:46:45.542605 26963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:46:45.542851 26963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:46:45.543205 26963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:46:45.543262 26963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:45.543306 26963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:46:45.543334 26963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:45.549646 26963 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.65:42411
I20260928 14:46:45.549791 27090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.65:42411 every 8 connection(s)
I20260928 14:46:45.550099 26963 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
I20260928 14:46:45.555572 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 26963
I20260928 14:46:45.555749 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance
I20260928 14:46:45.556006 9349 external_mini_cluster.cc:1428] Setting key b87a68b9786e3dde1eb8f7083a358294
I20260928 14:46:45.556703 26951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:45.550940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:42671 (local address 127.9.33.126:39343)
0928 14:46:45.551031 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:45.551033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:45.551288 (+ 255us) server_negotiation.cc:407] Connection header received
0928 14:46:45.551848 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:45.551850 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:45.551863 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:45.551928 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:45.552682 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.553369 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.554787 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.555039 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.555109 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:45.556335 (+ 1226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:45.556336 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:45.556337 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:46:45.556342 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:45.556359 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:46:45.556363 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:45.556436 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:45.556571 (+ 135us) server_negotiation.cc:299] Negotiation successful
0928 14:46:45.556616 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260928 14:46:45.556947 27093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:45.550928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:39343 (local address 127.9.33.65:42671)
0928 14:46:45.551122 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:45.551167 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:45.551746 (+ 579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:45.552002 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:45.552006 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:45.552015 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:46:45.552550 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:45.552555 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.553509 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.553510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:45.554678 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:45.554681 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.555149 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.555150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:45.555267 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:45.556037 (+ 770us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:46:45.556038 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:46:45.556086 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:46:45.556128 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:46:45.556457 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:46:45.556466 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:46:45.556471 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:46:45.556573 (+ 102us) client_negotiation.cc:240] Negotiation successful
0928 14:46:45.556703 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
I20260928 14:46:45.557054 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.66:0
--local_ip_for_outbound_sockets=127.9.33.66
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:39343
--builtin_ntp_servers=127.9.33.84:35779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 14:46:45.557704 27091 heartbeater.cc:347] Connected to a master server at 127.9.33.126:39343
I20260928 14:46:45.557796 27091 heartbeater.cc:464] Registering TS with master...
I20260928 14:46:45.558007 27091 heartbeater.cc:511] Master 127.9.33.126:39343 requested a full tablet report, sending...
I20260928 14:46:45.558593 26903 ts_manager.cc:194] Registered new tserver with Master: ae5c1445a5404ee2871e96504045bb37 (127.9.33.65:42411)
I20260928 14:46:45.559249 26903 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.33.65:42671
W20260928 14:46:45.642328 27094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:46:45.642584 27094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:46:45.642624 27094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:46:45.643975 27094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:46:45.644040 27094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:46:45.644088 27094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.66
I20260928 14:46:45.645601 27094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:35779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:39343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.66
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:46:45.645912 27094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:46:45.646250 27094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:46:45.651548 27102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:45.651602 27094 server_base.cc:1034] running on GCE node
W20260928 14:46:45.651592 27099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:45.651592 27100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:45.652101 27094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:46:45.652405 27094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:46:45.653576 27094 hybrid_clock.cc:648] HybridClock initialized: now 1790606805653560 us; error 32 us; skew 500 ppm
I20260928 14:46:45.655000 27094 webserver.cc:460] Webserver started at http://127.9.33.66:36307/ using document root <none> and password file <none>
I20260928 14:46:45.655293 27094 fs_manager.cc:359] Metadata directory not provided
I20260928 14:46:45.655369 27094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:46:45.655534 27094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:46:45.656723 27094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-1/data/instance:
uuid: "a5034e98628947cdac6e680a624d9d5f"
format_stamp: "Formatted at 2026-09-28 14:46:45 on dist-test-slave-pv0w"
server_key: "7592d285f952c7c2a6716e908233cdc0"
server_key_iv: "6079320fd983c6ee4b2426bdb7728f9a"
server_key_version: "encryptionkey@0"
I20260928 14:46:45.657106 27094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance:
uuid: "a5034e98628947cdac6e680a624d9d5f"
format_stamp: "Formatted at 2026-09-28 14:46:45 on dist-test-slave-pv0w"
server_key: "7592d285f952c7c2a6716e908233cdc0"
server_key_iv: "6079320fd983c6ee4b2426bdb7728f9a"
server_key_version: "encryptionkey@0"
I20260928 14:46:45.658461 27094 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260928 14:46:45.659268 27108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:45.659451 27094 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:46:45.659561 27094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-1/wal
uuid: "a5034e98628947cdac6e680a624d9d5f"
format_stamp: "Formatted at 2026-09-28 14:46:45 on dist-test-slave-pv0w"
server_key: "7592d285f952c7c2a6716e908233cdc0"
server_key_iv: "6079320fd983c6ee4b2426bdb7728f9a"
server_key_version: "encryptionkey@0"
I20260928 14:46:45.659655 27094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:46:45.673523 27094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:46:45.673868 27094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:46:45.674029 27094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:46:45.674284 27094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:46:45.674639 27094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:46:45.674698 27094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:45.674744 27094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:46:45.674777 27094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:45.681159 27094 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.66:38217
I20260928 14:46:45.681211 27221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.66:38217 every 8 connection(s)
I20260928 14:46:45.681614 27094 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
I20260928 14:46:45.682729 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 27094
I20260928 14:46:45.682847 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance
I20260928 14:46:45.683024 9349 external_mini_cluster.cc:1428] Setting key 5fb8f8afd378ede88c5b44baa819e7ea
I20260928 14:46:45.684019 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.67:0
--local_ip_for_outbound_sockets=127.9.33.67
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:39343
--builtin_ntp_servers=127.9.33.84:35779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 14:46:45.687463 26951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:45.682477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:44839 (local address 127.9.33.126:39343)
0928 14:46:45.682579 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:45.682583 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:45.682794 (+ 211us) server_negotiation.cc:407] Connection header received
0928 14:46:45.683311 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:45.683312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:45.683324 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:45.683387 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:45.684377 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.684880 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.685927 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.686135 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.686222 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:45.687143 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:45.687145 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:45.687146 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:46:45.687152 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:45.687167 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:46:45.687171 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:45.687238 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:45.687346 (+ 108us) server_negotiation.cc:299] Negotiation successful
0928 14:46:45.687381 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260928 14:46:45.687703 27224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:45.682442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:39343 (local address 127.9.33.66:44839)
0928 14:46:45.682646 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:45.682689 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:45.683184 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:45.683517 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:45.683521 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:45.683530 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:46:45.684266 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:45.684272 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.685000 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.685001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:45.685830 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:45.685832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.686235 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.686236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:45.686379 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:45.686977 (+ 598us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:46:45.686977 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:46:45.687009 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:46:45.687033 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:46:45.687253 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:46:45.687259 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:46:45.687262 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:46:45.687348 (+ 86us) client_negotiation.cc:240] Negotiation successful
0928 14:46:45.687470 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":86,"threads_started":1}
I20260928 14:46:45.688211 27222 heartbeater.cc:347] Connected to a master server at 127.9.33.126:39343
I20260928 14:46:45.688333 27222 heartbeater.cc:464] Registering TS with master...
I20260928 14:46:45.688560 27222 heartbeater.cc:511] Master 127.9.33.126:39343 requested a full tablet report, sending...
I20260928 14:46:45.688977 26903 ts_manager.cc:194] Registered new tserver with Master: a5034e98628947cdac6e680a624d9d5f (127.9.33.66:38217)
I20260928 14:46:45.689359 26903 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.33.66:44839
W20260928 14:46:45.769529 27225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:46:45.769761 27225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:46:45.769786 27225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:46:45.770947 27225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:46:45.770987 27225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:46:45.771013 27225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.67
I20260928 14:46:45.772418 27225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:35779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:39343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.67
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:46:45.772650 27225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:46:45.772926 27225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 14:46:45.778352 27225 server_base.cc:1034] running on GCE node
W20260928 14:46:45.778332 27233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:45.778332 27230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:45.778321 27231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:45.778843 27225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:46:45.779167 27225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:46:45.780347 27225 hybrid_clock.cc:648] HybridClock initialized: now 1790606805780328 us; error 37 us; skew 500 ppm
I20260928 14:46:45.781854 27225 webserver.cc:460] Webserver started at http://127.9.33.67:37107/ using document root <none> and password file <none>
I20260928 14:46:45.782159 27225 fs_manager.cc:359] Metadata directory not provided
I20260928 14:46:45.782238 27225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:46:45.782395 27225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:46:45.783555 27225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-2/data/instance:
uuid: "349d43bea592483eb1213d53d7f868c9"
format_stamp: "Formatted at 2026-09-28 14:46:45 on dist-test-slave-pv0w"
server_key: "0762488aa68b45c40a16afde85c50ec7"
server_key_iv: "d3013a5dc0b11b8ec5903f9ef55501ce"
server_key_version: "encryptionkey@0"
I20260928 14:46:45.783963 27225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance:
uuid: "349d43bea592483eb1213d53d7f868c9"
format_stamp: "Formatted at 2026-09-28 14:46:45 on dist-test-slave-pv0w"
server_key: "0762488aa68b45c40a16afde85c50ec7"
server_key_iv: "d3013a5dc0b11b8ec5903f9ef55501ce"
server_key_version: "encryptionkey@0"
I20260928 14:46:45.785375 27225 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:46:45.786249 27239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:45.786438 27225 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:46:45.786588 27225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-2/wal
uuid: "349d43bea592483eb1213d53d7f868c9"
format_stamp: "Formatted at 2026-09-28 14:46:45 on dist-test-slave-pv0w"
server_key: "0762488aa68b45c40a16afde85c50ec7"
server_key_iv: "d3013a5dc0b11b8ec5903f9ef55501ce"
server_key_version: "encryptionkey@0"
I20260928 14:46:45.786675 27225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:46:45.797487 27225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:46:45.797987 27225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:46:45.798173 27225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:46:45.798753 27225 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:46:45.799190 27225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:46:45.799258 27225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:45.799297 27225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:46:45.799341 27225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:45.805266 27225 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.67:37779
I20260928 14:46:45.805308 27352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.67:37779 every 8 connection(s)
I20260928 14:46:45.805684 27225 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
I20260928 14:46:45.809177 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 27225
I20260928 14:46:45.809266 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance
I20260928 14:46:45.809465 9349 external_mini_cluster.cc:1428] Setting key 2d4862a08ca16fee203c85f4afef24ed
I20260928 14:46:45.811352 26951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:45.806399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:37047 (local address 127.9.33.126:39343)
0928 14:46:45.806492 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:45.806494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:45.806719 (+ 225us) server_negotiation.cc:407] Connection header received
0928 14:46:45.807229 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:45.807231 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:45.807242 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:45.807298 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:45.808089 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.808604 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.809814 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.810128 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.810208 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:45.811106 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:45.811107 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:45.811107 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:46:45.811112 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:45.811129 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:46:45.811133 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:45.811179 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:45.811263 (+ 84us) server_negotiation.cc:299] Negotiation successful
0928 14:46:45.811291 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260928 14:46:45.811597 27355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:45.806364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:39343 (local address 127.9.33.67:37047)
0928 14:46:45.806591 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:45.806627 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:45.807138 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:45.807373 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:45.807377 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:45.807385 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:46:45.807988 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:45.807994 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.808704 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.808705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:45.809727 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:45.809731 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.810234 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.810235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:45.810405 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:45.810954 (+ 549us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:46:45.810954 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:46:45.810984 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:46:45.811020 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:46:45.811190 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:46:45.811194 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:46:45.811198 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:46:45.811252 (+ 54us) client_negotiation.cc:240] Negotiation successful
0928 14:46:45.811338 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":110,"threads_started":1}
I20260928 14:46:45.812160 27353 heartbeater.cc:347] Connected to a master server at 127.9.33.126:39343
I20260928 14:46:45.812258 27353 heartbeater.cc:464] Registering TS with master...
I20260928 14:46:45.812415 27353 heartbeater.cc:511] Master 127.9.33.126:39343 requested a full tablet report, sending...
I20260928 14:46:45.812840 26903 ts_manager.cc:194] Registered new tserver with Master: 349d43bea592483eb1213d53d7f868c9 (127.9.33.67:37779)
I20260928 14:46:45.813233 26903 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.33.67:37047
I20260928 14:46:45.821501 9349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:46:45.826431 26951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:45.823031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44522 (local address 127.9.33.126:39343)
0928 14:46:45.823131 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:45.823133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:45.823243 (+ 110us) server_negotiation.cc:407] Connection header received
0928 14:46:45.823341 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:45.823342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:45.823358 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:45.823435 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:45.823617 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.824247 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.825110 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.825321 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.825408 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:45.826122 (+ 714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:45.826124 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:45.826124 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:46:45.826129 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:45.826152 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:46:45.826156 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:45.826231 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:45.826303 (+ 72us) server_negotiation.cc:299] Negotiation successful
0928 14:46:45.826335 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260928 14:46:45.828356 26903 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44522:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:46:45.830009 26903 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:46:45.836521 27362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:45.832406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:42411 (local address 127.0.0.1:45880)
0928 14:46:45.832711 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:45.832754 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:45.832858 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:45.833135 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:45.833138 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:45.833141 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:46:45.833226 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:45.833228 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.834218 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.834219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:45.835347 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:45.835349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.836196 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.836197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:45.836285 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:45.836299 (+ 14us) client_negotiation.cc:769] Sending connection context
0928 14:46:45.836377 (+ 78us) client_negotiation.cc:240] Negotiation successful
0928 14:46:45.836435 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":62,"threads_started":1}
I20260928 14:46:45.836534 27364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:45.832494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45880 (local address 127.9.33.65:42411)
0928 14:46:45.832712 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:45.832718 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:45.832845 (+ 127us) server_negotiation.cc:407] Connection header received
0928 14:46:45.832951 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:45.832954 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:45.832970 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:45.833056 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:46:45.833314 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.834125 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.835456 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.836100 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.836195 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:45.836297 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:45.836393 (+ 96us) server_negotiation.cc:299] Negotiation successful
0928 14:46:45.836435 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":105,"threads_started":1}
I20260928 14:46:45.837250 27363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:45.832437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:37779 (local address 127.0.0.1:40656)
0928 14:46:45.832989 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:45.833000 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:45.833043 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:45.833313 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:45.833314 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:45.833316 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:46:45.833368 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:45.833371 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.834731 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.834732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:45.835708 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:45.835710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.837020 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.837021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:45.837087 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:45.837090 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:46:45.837137 (+ 47us) client_negotiation.cc:240] Negotiation successful
0928 14:46:45.837179 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"mutex_wait_us":3,"thread_start_us":156,"threads_started":1}
I20260928 14:46:45.837478 27365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:45.832738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40656 (local address 127.9.33.67:37779)
0928 14:46:45.832920 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:45.832925 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:45.833104 (+ 179us) server_negotiation.cc:407] Connection header received
0928 14:46:45.833152 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:45.833154 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:45.833169 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:45.833240 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:46:45.833453 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.834623 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.835844 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.836926 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.837016 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:45.837109 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:45.837184 (+ 75us) server_negotiation.cc:299] Negotiation successful
0928 14:46:45.837227 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":79,"threads_started":1}
I20260928 14:46:45.837363 27025 tablet_service.cc:1467] Processing CreateTablet for tablet df0a0f87c85f419381c1b0618f2eb35a (DEFAULT_TABLE table=test-table [id=56165504075341a2a7e14e45ed64ca84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:46:45.837813 27025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df0a0f87c85f419381c1b0618f2eb35a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:46:45.838281 27361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:45.832437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:38217 (local address 127.0.0.1:38402)
0928 14:46:45.832980 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:45.832989 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:45.833155 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:45.833420 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:45.833421 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:45.833423 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:46:45.833463 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:45.833464 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.835798 (+ 2334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.835799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:45.836874 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:45.836877 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.838003 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.838004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:45.838074 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:45.838078 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:46:45.838145 (+ 67us) client_negotiation.cc:240] Negotiation successful
0928 14:46:45.838193 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":160,"threads_started":1}
I20260928 14:46:45.838294 27366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:45.832738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38402 (local address 127.9.33.66:38217)
0928 14:46:45.832920 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:45.832925 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:45.833083 (+ 158us) server_negotiation.cc:407] Connection header received
0928 14:46:45.833258 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:45.833261 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:45.833278 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:45.833352 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:46:45.833528 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.834640 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.836984 (+ 2344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:45.837891 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:45.838003 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:45.838098 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:45.838160 (+ 62us) server_negotiation.cc:299] Negotiation successful
0928 14:46:45.838202 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":79,"threads_started":1}
I20260928 14:46:45.838233 27287 tablet_service.cc:1467] Processing CreateTablet for tablet df0a0f87c85f419381c1b0618f2eb35a (DEFAULT_TABLE table=test-table [id=56165504075341a2a7e14e45ed64ca84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:46:45.838635 27287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df0a0f87c85f419381c1b0618f2eb35a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:46:45.839376 27156 tablet_service.cc:1467] Processing CreateTablet for tablet df0a0f87c85f419381c1b0618f2eb35a (DEFAULT_TABLE table=test-table [id=56165504075341a2a7e14e45ed64ca84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:46:45.839779 27156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df0a0f87c85f419381c1b0618f2eb35a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:46:45.841383 27367 tablet_bootstrap.cc:492] T df0a0f87c85f419381c1b0618f2eb35a P 349d43bea592483eb1213d53d7f868c9: Bootstrap starting.
I20260928 14:46:45.841835 27368 tablet_bootstrap.cc:492] T df0a0f87c85f419381c1b0618f2eb35a P ae5c1445a5404ee2871e96504045bb37: Bootstrap starting.
I20260928 14:46:45.842525 27367 tablet_bootstrap.cc:654] T df0a0f87c85f419381c1b0618f2eb35a P 349d43bea592483eb1213d53d7f868c9: Neither blocks nor log segments found. Creating new log.
I20260928 14:46:45.842525 27368 tablet_bootstrap.cc:654] T df0a0f87c85f419381c1b0618f2eb35a P ae5c1445a5404ee2871e96504045bb37: Neither blocks nor log segments found. Creating new log.
I20260928 14:46:45.842902 27367 log.cc:824] T df0a0f87c85f419381c1b0618f2eb35a P 349d43bea592483eb1213d53d7f868c9: Log is configured to *not* fsync() on all Append() calls
I20260928 14:46:45.843189 27369 tablet_bootstrap.cc:492] T df0a0f87c85f419381c1b0618f2eb35a P a5034e98628947cdac6e680a624d9d5f: Bootstrap starting.
I20260928 14:46:45.843993 27369 tablet_bootstrap.cc:654] T df0a0f87c85f419381c1b0618f2eb35a P a5034e98628947cdac6e680a624d9d5f: Neither blocks nor log segments found. Creating new log.
I20260928 14:46:45.844044 27367 tablet_bootstrap.cc:492] T df0a0f87c85f419381c1b0618f2eb35a P 349d43bea592483eb1213d53d7f868c9: No bootstrap required, opened a new log
I20260928 14:46:45.844115 27367 ts_tablet_manager.cc:1397] T df0a0f87c85f419381c1b0618f2eb35a P 349d43bea592483eb1213d53d7f868c9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260928 14:46:45.842902 27368 log.cc:824] T df0a0f87c85f419381c1b0618f2eb35a P ae5c1445a5404ee2871e96504045bb37: Log is configured to *not* fsync() on all Append() calls
I20260928 14:46:45.844369 27369 log.cc:824] T df0a0f87c85f419381c1b0618f2eb35a P a5034e98628947cdac6e680a624d9d5f: Log is configured to *not* fsync() on all Append() calls
I20260928 14:46:45.844962 27368 tablet_bootstrap.cc:492] T df0a0f87c85f419381c1b0618f2eb35a P ae5c1445a5404ee2871e96504045bb37: No bootstrap required, opened a new log
I20260928 14:46:45.845050 27368 ts_tablet_manager.cc:1397] T df0a0f87c85f419381c1b0618f2eb35a P ae5c1445a5404ee2871e96504045bb37: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260928 14:46:45.847071 27367 raft_consensus.cc:348] T df0a0f87c85f419381c1b0618f2eb35a P 349d43bea592483eb1213d53d7f868c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae5c1445a5404ee2871e96504045bb37" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 42411 } } peers { permanent_uuid: "a5034e98628947cdac6e680a624d9d5f" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38217 } } peers { permanent_uuid: "349d43bea592483eb1213d53d7f868c9" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 37779 } }
I20260928 14:46:45.847268 27367 raft_consensus.cc:374] T df0a0f87c85f419381c1b0618f2eb35a P 349d43bea592483eb1213d53d7f868c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:46:45.847312 27367 raft_consensus.cc:729] T df0a0f87c85f419381c1b0618f2eb35a P 349d43bea592483eb1213d53d7f868c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 349d43bea592483eb1213d53d7f868c9, State: Initialized, Role: FOLLOWER
I20260928 14:46:45.847450 27367 consensus_queue.cc:260] T df0a0f87c85f419381c1b0618f2eb35a P 349d43bea592483eb1213d53d7f868c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae5c1445a5404ee2871e96504045bb37" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 42411 } } peers { permanent_uuid: "a5034e98628947cdac6e680a624d9d5f" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38217 } } peers { permanent_uuid: "349d43bea592483eb1213d53d7f868c9" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 37779 } }
I20260928 14:46:45.847826 27353 heartbeater.cc:503] Master 127.9.33.126:39343 was elected leader, sending a full tablet report...
I20260928 14:46:45.847767 27368 raft_consensus.cc:348] T df0a0f87c85f419381c1b0618f2eb35a P ae5c1445a5404ee2871e96504045bb37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae5c1445a5404ee2871e96504045bb37" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 42411 } } peers { permanent_uuid: "a5034e98628947cdac6e680a624d9d5f" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38217 } } peers { permanent_uuid: "349d43bea592483eb1213d53d7f868c9" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 37779 } }
I20260928 14:46:45.847936 27368 raft_consensus.cc:374] T df0a0f87c85f419381c1b0618f2eb35a P ae5c1445a5404ee2871e96504045bb37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:46:45.847980 27368 raft_consensus.cc:729] T df0a0f87c85f419381c1b0618f2eb35a P ae5c1445a5404ee2871e96504045bb37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae5c1445a5404ee2871e96504045bb37, State: Initialized, Role: FOLLOWER
I20260928 14:46:45.848085 27367 ts_tablet_manager.cc:1428] T df0a0f87c85f419381c1b0618f2eb35a P 349d43bea592483eb1213d53d7f868c9: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260928 14:46:45.848098 27368 consensus_queue.cc:260] T df0a0f87c85f419381c1b0618f2eb35a P ae5c1445a5404ee2871e96504045bb37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae5c1445a5404ee2871e96504045bb37" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 42411 } } peers { permanent_uuid: "a5034e98628947cdac6e680a624d9d5f" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38217 } } peers { permanent_uuid: "349d43bea592483eb1213d53d7f868c9" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 37779 } }
I20260928 14:46:45.848363 27368 ts_tablet_manager.cc:1428] T df0a0f87c85f419381c1b0618f2eb35a P ae5c1445a5404ee2871e96504045bb37: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260928 14:46:45.848431 27091 heartbeater.cc:503] Master 127.9.33.126:39343 was elected leader, sending a full tablet report...
I20260928 14:46:45.849164 27369 tablet_bootstrap.cc:492] T df0a0f87c85f419381c1b0618f2eb35a P a5034e98628947cdac6e680a624d9d5f: No bootstrap required, opened a new log
I20260928 14:46:45.849244 27369 ts_tablet_manager.cc:1397] T df0a0f87c85f419381c1b0618f2eb35a P a5034e98628947cdac6e680a624d9d5f: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260928 14:46:45.851305 27369 raft_consensus.cc:348] T df0a0f87c85f419381c1b0618f2eb35a P a5034e98628947cdac6e680a624d9d5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae5c1445a5404ee2871e96504045bb37" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 42411 } } peers { permanent_uuid: "a5034e98628947cdac6e680a624d9d5f" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38217 } } peers { permanent_uuid: "349d43bea592483eb1213d53d7f868c9" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 37779 } }
I20260928 14:46:45.851477 27369 raft_consensus.cc:374] T df0a0f87c85f419381c1b0618f2eb35a P a5034e98628947cdac6e680a624d9d5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:46:45.851526 27369 raft_consensus.cc:729] T df0a0f87c85f419381c1b0618f2eb35a P a5034e98628947cdac6e680a624d9d5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5034e98628947cdac6e680a624d9d5f, State: Initialized, Role: FOLLOWER
I20260928 14:46:45.851660 27369 consensus_queue.cc:260] T df0a0f87c85f419381c1b0618f2eb35a P a5034e98628947cdac6e680a624d9d5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae5c1445a5404ee2871e96504045bb37" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 42411 } } peers { permanent_uuid: "a5034e98628947cdac6e680a624d9d5f" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38217 } } peers { permanent_uuid: "349d43bea592483eb1213d53d7f868c9" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 37779 } }
I20260928 14:46:45.851990 27369 ts_tablet_manager.cc:1428] T df0a0f87c85f419381c1b0618f2eb35a P a5034e98628947cdac6e680a624d9d5f: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260928 14:46:45.852135 27222 heartbeater.cc:503] Master 127.9.33.126:39343 was elected leader, sending a full tablet report...
W20260928 14:46:45.932238 27223 tablet.cc:2369] T df0a0f87c85f419381c1b0618f2eb35a P a5034e98628947cdac6e680a624d9d5f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:46:46.028280 27373 raft_consensus.cc:482] T df0a0f87c85f419381c1b0618f2eb35a P 349d43bea592483eb1213d53d7f868c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:46:46.028422 27373 raft_consensus.cc:504] T df0a0f87c85f419381c1b0618f2eb35a P 349d43bea592483eb1213d53d7f868c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae5c1445a5404ee2871e96504045bb37" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 42411 } } peers { permanent_uuid: "a5034e98628947cdac6e680a624d9d5f" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38217 } } peers { permanent_uuid: "349d43bea592483eb1213d53d7f868c9" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 37779 } }
I20260928 14:46:46.028949 27373 leader_election.cc:288] T df0a0f87c85f419381c1b0618f2eb35a P 349d43bea592483eb1213d53d7f868c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae5c1445a5404ee2871e96504045bb37 (127.9.33.65:42411), a5034e98628947cdac6e680a624d9d5f (127.9.33.66:38217)
I20260928 14:46:46.032055 27366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:46.029240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:43009 (local address 127.9.33.66:38217)
0928 14:46:46.029291 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:46.029293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:46.029303 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:46:46.029416 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:46.029417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:46.029429 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:46.029490 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:46:46.029694 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:46.030237 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:46.031232 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:46.031652 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:46.031709 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:46.031768 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:46.031920 (+ 152us) server_negotiation.cc:299] Negotiation successful
0928 14:46:46.031972 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260928 14:46:46.032073 27355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:46.029167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:38217 (local address 127.9.33.67:43009)
0928 14:46:46.029236 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:46.029251 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:46.029307 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:46.029524 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:46.029526 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:46.029528 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:46:46.029606 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:46.029608 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:46.030328 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:46.030329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:46.031137 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:46.031140 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:46.031758 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:46.031759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:46.031840 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:46.031844 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:46:46.031894 (+ 50us) client_negotiation.cc:240] Negotiation successful
0928 14:46:46.031955 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260928 14:46:46.032085 27364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:46.029213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:48337 (local address 127.9.33.65:42411)
0928 14:46:46.029260 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:46.029262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:46.029534 (+ 272us) server_negotiation.cc:407] Connection header received
0928 14:46:46.029594 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:46.029595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:46.029605 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:46.029661 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:46:46.029831 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:46.030336 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:46.031299 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:46.031662 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:46.031770 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:46.031780 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:46.031970 (+ 190us) server_negotiation.cc:299] Negotiation successful
0928 14:46:46.032022 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260928 14:46:46.032102 27376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:46.029168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:42411 (local address 127.9.33.67:48337)
0928 14:46:46.029457 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:46.029468 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:46.029527 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:46.029693 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:46.029694 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:46.029696 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:46:46.029762 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:46.029763 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:46.030430 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:46.030430 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:46.031204 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:46.031207 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:46.031800 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:46.031801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:46.031872 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:46.031875 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:46:46.031952 (+ 77us) client_negotiation.cc:240] Negotiation successful
0928 14:46:46.031987 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":103,"thread_start_us":110,"threads_started":1}
I20260928 14:46:46.032449 27045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df0a0f87c85f419381c1b0618f2eb35a" candidate_uuid: "349d43bea592483eb1213d53d7f868c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae5c1445a5404ee2871e96504045bb37" is_pre_election: true
I20260928 14:46:46.032476 27176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df0a0f87c85f419381c1b0618f2eb35a" candidate_uuid: "349d43bea592483eb1213d53d7f868c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5034e98628947cdac6e680a624d9d5f" is_pre_election: true
I20260928 14:46:46.032646 27176 raft_consensus.cc:2454] T df0a0f87c85f419381c1b0618f2eb35a P a5034e98628947cdac6e680a624d9d5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 349d43bea592483eb1213d53d7f868c9 in term 0.
I20260928 14:46:46.032656 27045 raft_consensus.cc:2454] T df0a0f87c85f419381c1b0618f2eb35a P ae5c1445a5404ee2871e96504045bb37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 349d43bea592483eb1213d53d7f868c9 in term 0.
I20260928 14:46:46.032923 27242 leader_election.cc:302] T df0a0f87c85f419381c1b0618f2eb35a P 349d43bea592483eb1213d53d7f868c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 349d43bea592483eb1213d53d7f868c9, a5034e98628947cdac6e680a624d9d5f; no voters:
I20260928 14:46:46.033090 27373 raft_consensus.cc:2781] T df0a0f87c85f419381c1b0618f2eb35a P 349d43bea592483eb1213d53d7f868c9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:46:46.033175 27373 raft_consensus.cc:482] T df0a0f87c85f419381c1b0618f2eb35a P 349d43bea592483eb1213d53d7f868c9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:46:46.033211 27373 raft_consensus.cc:3037] T df0a0f87c85f419381c1b0618f2eb35a P 349d43bea592483eb1213d53d7f868c9 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:46:46.033932 27373 raft_consensus.cc:504] T df0a0f87c85f419381c1b0618f2eb35a P 349d43bea592483eb1213d53d7f868c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae5c1445a5404ee2871e96504045bb37" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 42411 } } peers { permanent_uuid: "a5034e98628947cdac6e680a624d9d5f" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38217 } } peers { permanent_uuid: "349d43bea592483eb1213d53d7f868c9" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 37779 } }
I20260928 14:46:46.034098 27373 leader_election.cc:288] T df0a0f87c85f419381c1b0618f2eb35a P 349d43bea592483eb1213d53d7f868c9 [CANDIDATE]: Term 1 election: Requested vote from peers ae5c1445a5404ee2871e96504045bb37 (127.9.33.65:42411), a5034e98628947cdac6e680a624d9d5f (127.9.33.66:38217)
I20260928 14:46:46.034250 27176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df0a0f87c85f419381c1b0618f2eb35a" candidate_uuid: "349d43bea592483eb1213d53d7f868c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5034e98628947cdac6e680a624d9d5f"
I20260928 14:46:46.034273 27045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df0a0f87c85f419381c1b0618f2eb35a" candidate_uuid: "349d43bea592483eb1213d53d7f868c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae5c1445a5404ee2871e96504045bb37"
I20260928 14:46:46.034340 27176 raft_consensus.cc:3037] T df0a0f87c85f419381c1b0618f2eb35a P a5034e98628947cdac6e680a624d9d5f [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:46:46.034365 27045 raft_consensus.cc:3037] T df0a0f87c85f419381c1b0618f2eb35a P ae5c1445a5404ee2871e96504045bb37 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:46:46.035101 27045 raft_consensus.cc:2454] T df0a0f87c85f419381c1b0618f2eb35a P ae5c1445a5404ee2871e96504045bb37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 349d43bea592483eb1213d53d7f868c9 in term 1.
I20260928 14:46:46.035130 27176 raft_consensus.cc:2454] T df0a0f87c85f419381c1b0618f2eb35a P a5034e98628947cdac6e680a624d9d5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 349d43bea592483eb1213d53d7f868c9 in term 1.
I20260928 14:46:46.035292 27241 leader_election.cc:302] T df0a0f87c85f419381c1b0618f2eb35a P 349d43bea592483eb1213d53d7f868c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 349d43bea592483eb1213d53d7f868c9, ae5c1445a5404ee2871e96504045bb37; no voters:
I20260928 14:46:46.035396 27373 raft_consensus.cc:2781] T df0a0f87c85f419381c1b0618f2eb35a P 349d43bea592483eb1213d53d7f868c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:46:46.035598 27373 raft_consensus.cc:686] T df0a0f87c85f419381c1b0618f2eb35a P 349d43bea592483eb1213d53d7f868c9 [term 1 LEADER]: Becoming Leader. State: Replica: 349d43bea592483eb1213d53d7f868c9, State: Running, Role: LEADER
I20260928 14:46:46.035712 27373 consensus_queue.cc:237] T df0a0f87c85f419381c1b0618f2eb35a P 349d43bea592483eb1213d53d7f868c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae5c1445a5404ee2871e96504045bb37" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 42411 } } peers { permanent_uuid: "a5034e98628947cdac6e680a624d9d5f" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38217 } } peers { permanent_uuid: "349d43bea592483eb1213d53d7f868c9" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 37779 } }
I20260928 14:46:46.036695 26903 catalog_manager.cc:5495] T df0a0f87c85f419381c1b0618f2eb35a P 349d43bea592483eb1213d53d7f868c9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 349d43bea592483eb1213d53d7f868c9 (127.9.33.67). New cstate: current_term: 1 leader_uuid: "349d43bea592483eb1213d53d7f868c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae5c1445a5404ee2871e96504045bb37" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 42411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5034e98628947cdac6e680a624d9d5f" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "349d43bea592483eb1213d53d7f868c9" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 37779 } health_report { overall_health: HEALTHY } } }
W20260928 14:46:46.050913 27092 tablet.cc:2369] T df0a0f87c85f419381c1b0618f2eb35a P ae5c1445a5404ee2871e96504045bb37: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:46:46.056236 27354 tablet.cc:2369] T df0a0f87c85f419381c1b0618f2eb35a P 349d43bea592483eb1213d53d7f868c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:46:46.065392 27365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:46.062893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40670 (local address 127.9.33.67:37779)
0928 14:46:46.062956 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:46.062958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:46.062966 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:46:46.063039 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:46.063039 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:46.063058 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:46.063126 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:46:46.063280 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:46.063808 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:46.064550 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:46.064746 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:46.064842 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:46.064952 (+ 110us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:46:46.065070 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:46:46.065139 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:46.065236 (+ 97us) server_negotiation.cc:299] Negotiation successful
0928 14:46:46.065254 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260928 14:46:46.067476 27045 raft_consensus.cc:1261] T df0a0f87c85f419381c1b0618f2eb35a P ae5c1445a5404ee2871e96504045bb37 [term 1 FOLLOWER]: Refusing update from remote peer 349d43bea592483eb1213d53d7f868c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:46:46.067507 27176 raft_consensus.cc:1261] T df0a0f87c85f419381c1b0618f2eb35a P a5034e98628947cdac6e680a624d9d5f [term 1 FOLLOWER]: Refusing update from remote peer 349d43bea592483eb1213d53d7f868c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:46:46.067940 27377 consensus_queue.cc:1035] T df0a0f87c85f419381c1b0618f2eb35a P 349d43bea592483eb1213d53d7f868c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae5c1445a5404ee2871e96504045bb37" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 42411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:46:46.068091 27373 consensus_queue.cc:1035] T df0a0f87c85f419381c1b0618f2eb35a P 349d43bea592483eb1213d53d7f868c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5034e98628947cdac6e680a624d9d5f" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 38217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:46:46.070936 27382 mvcc.cc:205] Tried to move back new op lower bound from 7334325477649186816 to 7334325477523177472. Current Snapshot: MvccSnapshot[applied={T|T < 7334325477649186816}]
I20260928 14:46:46.071753 27381 mvcc.cc:205] Tried to move back new op lower bound from 7334325477649186816 to 7334325477523177472. Current Snapshot: MvccSnapshot[applied={T|T < 7334325477649186816}]
I20260928 14:46:46.072996 26903 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:46:46.073100 26903 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:46:46.073900 26903 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 69ea972f32794056be7cc25ccac94002: Sending DeleteTablet for 3 replicas of tablet df0a0f87c85f419381c1b0618f2eb35a
I20260928 14:46:46.074255 27156 tablet_service.cc:1514] Processing DeleteTablet for tablet df0a0f87c85f419381c1b0618f2eb35a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:46:46 UTC) from {username='slave'} at 127.0.0.1:38402
I20260928 14:46:46.074261 27025 tablet_service.cc:1514] Processing DeleteTablet for tablet df0a0f87c85f419381c1b0618f2eb35a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:46:46 UTC) from {username='slave'} at 127.0.0.1:45880
I20260928 14:46:46.074518 27390 tablet_replica.cc:330] T df0a0f87c85f419381c1b0618f2eb35a P a5034e98628947cdac6e680a624d9d5f: stopping tablet replica
I20260928 14:46:46.074515 27389 tablet_replica.cc:330] T df0a0f87c85f419381c1b0618f2eb35a P ae5c1445a5404ee2871e96504045bb37: stopping tablet replica
I20260928 14:46:46.074630 27390 raft_consensus.cc:2229] T df0a0f87c85f419381c1b0618f2eb35a P a5034e98628947cdac6e680a624d9d5f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:46:46.074743 27389 raft_consensus.cc:2229] T df0a0f87c85f419381c1b0618f2eb35a P ae5c1445a5404ee2871e96504045bb37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:46:46.074750 27390 raft_consensus.cc:2258] T df0a0f87c85f419381c1b0618f2eb35a P a5034e98628947cdac6e680a624d9d5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:46:46.074725 27287 tablet_service.cc:1514] Processing DeleteTablet for tablet df0a0f87c85f419381c1b0618f2eb35a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:46:46 UTC) from {username='slave'} at 127.0.0.1:40656
I20260928 14:46:46.075127 27391 tablet_replica.cc:330] T df0a0f87c85f419381c1b0618f2eb35a P 349d43bea592483eb1213d53d7f868c9: stopping tablet replica
I20260928 14:46:46.075124 27389 raft_consensus.cc:2258] T df0a0f87c85f419381c1b0618f2eb35a P ae5c1445a5404ee2871e96504045bb37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:46:46.075222 27391 raft_consensus.cc:2229] T df0a0f87c85f419381c1b0618f2eb35a P 349d43bea592483eb1213d53d7f868c9 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:46:46.075218 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 26963
I20260928 14:46:46.075322 27391 raft_consensus.cc:2258] T df0a0f87c85f419381c1b0618f2eb35a P 349d43bea592483eb1213d53d7f868c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:46:46.076289 27391 ts_tablet_manager.cc:1905] T df0a0f87c85f419381c1b0618f2eb35a P 349d43bea592483eb1213d53d7f868c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:46:46.076710 27390 ts_tablet_manager.cc:1905] T df0a0f87c85f419381c1b0618f2eb35a P a5034e98628947cdac6e680a624d9d5f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:46:46.078151 27391 ts_tablet_manager.cc:1918] T df0a0f87c85f419381c1b0618f2eb35a P 349d43bea592483eb1213d53d7f868c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:46:46.078245 27391 log.cc:1196] T df0a0f87c85f419381c1b0618f2eb35a P 349d43bea592483eb1213d53d7f868c9: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-2/wal/wals/df0a0f87c85f419381c1b0618f2eb35a
I20260928 14:46:46.078485 27391 ts_tablet_manager.cc:1939] T df0a0f87c85f419381c1b0618f2eb35a P 349d43bea592483eb1213d53d7f868c9: Deleting consensus metadata
I20260928 14:46:46.078687 27390 ts_tablet_manager.cc:1918] T df0a0f87c85f419381c1b0618f2eb35a P a5034e98628947cdac6e680a624d9d5f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:46:46.078765 27390 log.cc:1196] T df0a0f87c85f419381c1b0618f2eb35a P a5034e98628947cdac6e680a624d9d5f: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790606561545179-9349-0/minicluster-data/ts-1/wal/wals/df0a0f87c85f419381c1b0618f2eb35a
I20260928 14:46:46.079008 27390 ts_tablet_manager.cc:1939] T df0a0f87c85f419381c1b0618f2eb35a P a5034e98628947cdac6e680a624d9d5f: Deleting consensus metadata
I20260928 14:46:46.078987 26890 catalog_manager.cc:4841] TS 349d43bea592483eb1213d53d7f868c9 (127.9.33.67:37779): tablet df0a0f87c85f419381c1b0618f2eb35a (table test-table [id=56165504075341a2a7e14e45ed64ca84]) successfully deleted
I20260928 14:46:46.079418 26889 catalog_manager.cc:4841] TS a5034e98628947cdac6e680a624d9d5f (127.9.33.66:38217): tablet df0a0f87c85f419381c1b0618f2eb35a (table test-table [id=56165504075341a2a7e14e45ed64ca84]) successfully deleted
W20260928 14:46:46.081923 26888 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.33.65:42411 (error 108)
I20260928 14:46:46.082466 27361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:46.082324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:42411 (local address 127.0.0.1:45890)
0928 14:46:46.082378 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:46.082420 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.65:42411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260928 14:46:46.082537 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 27094
W20260928 14:46:46.084584 26888 catalog_manager.cc:4584] TS ae5c1445a5404ee2871e96504045bb37 (127.9.33.65:42411): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet df0a0f87c85f419381c1b0618f2eb35a: Network error: Client connection negotiation failed: client connection to 127.9.33.65:42411: connect: Connection refused (error 111)
I20260928 14:46:46.091310 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 27225
I20260928 14:46:46.096516 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 26872
2026-09-28T14:46:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (866 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:46:46 dist-test-slave-pv0w krb5kdc[27395](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:46:46 dist-test-slave-pv0w krb5kdc[27395](info): set up 2 sockets
Sep 28 14:46:46 dist-test-slave-pv0w krb5kdc[27395](info): commencing operation
krb5kdc: starting...
W20260928 14:46:47.136219 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:46:47 dist-test-slave-pv0w krb5kdc[27395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606807, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:46:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:46:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.33.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
I20260928 14:46:47.264133 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:35711
--webserver_interface=127.9.33.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:40423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:35711
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:46:47.348238 27408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:46:47.348488 27408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:46:47.348515 27408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:46:47.349813 27408 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:46:47.349869 27408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:46:47.349884 27408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:46:47.349900 27408 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:46:47.349911 27408 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:46:47.351552 27408 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:40423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:35711
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:35711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:46:47.351867 27408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:46:47.352174 27408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:46:47.358045 27416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:47.358103 27413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:47.358054 27414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:47.358379 27408 server_base.cc:1034] running on GCE node
I20260928 14:46:47.358706 27408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:46:47.359036 27408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:46:47.360186 27408 hybrid_clock.cc:648] HybridClock initialized: now 1790606807360181 us; error 37 us; skew 500 ppm
Sep 28 14:46:47 dist-test-slave-pv0w krb5kdc[27395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606807, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:46:47.363129 27408 init.cc:377] Logged in from keytab as kudu/127.9.33.126@KRBTEST.COM (short username kudu)
I20260928 14:46:47.363682 27408 webserver.cc:460] Webserver started at http://127.9.33.126:36883/ using document root <none> and password file <none>
I20260928 14:46:47.363952 27408 fs_manager.cc:359] Metadata directory not provided
I20260928 14:46:47.364025 27408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:46:47.364193 27408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:46:47.365327 27408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "964c81b7db814657a0df6455f493f626"
format_stamp: "Formatted at 2026-09-28 14:46:47 on dist-test-slave-pv0w"
server_key: "7d4cb9a4777b0e5f222f20b33c58bdcb"
server_key_iv: "2f86acb99ba06a64ec804edc28e0bad6"
server_key_version: "encryptionkey@0"
I20260928 14:46:47.365728 27408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "964c81b7db814657a0df6455f493f626"
format_stamp: "Formatted at 2026-09-28 14:46:47 on dist-test-slave-pv0w"
server_key: "7d4cb9a4777b0e5f222f20b33c58bdcb"
server_key_iv: "2f86acb99ba06a64ec804edc28e0bad6"
server_key_version: "encryptionkey@0"
I20260928 14:46:47.367134 27408 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:46:47.368075 27423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:47.368259 27408 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:46:47.368369 27408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "964c81b7db814657a0df6455f493f626"
format_stamp: "Formatted at 2026-09-28 14:46:47 on dist-test-slave-pv0w"
server_key: "7d4cb9a4777b0e5f222f20b33c58bdcb"
server_key_iv: "2f86acb99ba06a64ec804edc28e0bad6"
server_key_version: "encryptionkey@0"
I20260928 14:46:47.368450 27408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:46:47.379078 27408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:46:47.381665 27408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:46:47.381845 27408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:46:47.386236 27408 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:35711
I20260928 14:46:47.386279 27475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:35711 every 8 connection(s)
I20260928 14:46:47.386664 27408 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:46:47.387584 27476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:46:47.389534 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 27408
I20260928 14:46:47.389674 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:46:47.389866 9349 external_mini_cluster.cc:1428] Setting key 5766938e5d51247508050a99167297e1
I20260928 14:46:47.390251 27476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 964c81b7db814657a0df6455f493f626: Bootstrap starting.
I20260928 14:46:47.390926 27476 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 964c81b7db814657a0df6455f493f626: Neither blocks nor log segments found. Creating new log.
I20260928 14:46:47.391240 27476 log.cc:824] T 00000000000000000000000000000000 P 964c81b7db814657a0df6455f493f626: Log is configured to *not* fsync() on all Append() calls
I20260928 14:46:47.392011 27476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 964c81b7db814657a0df6455f493f626: No bootstrap required, opened a new log
I20260928 14:46:47.394109 27476 raft_consensus.cc:348] T 00000000000000000000000000000000 P 964c81b7db814657a0df6455f493f626 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "964c81b7db814657a0df6455f493f626" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35711 } }
I20260928 14:46:47.394372 27476 raft_consensus.cc:374] T 00000000000000000000000000000000 P 964c81b7db814657a0df6455f493f626 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:46:47.394426 27476 raft_consensus.cc:729] T 00000000000000000000000000000000 P 964c81b7db814657a0df6455f493f626 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 964c81b7db814657a0df6455f493f626, State: Initialized, Role: FOLLOWER
I20260928 14:46:47.394569 27476 consensus_queue.cc:260] T 00000000000000000000000000000000 P 964c81b7db814657a0df6455f493f626 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "964c81b7db814657a0df6455f493f626" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35711 } }
I20260928 14:46:47.394665 27476 raft_consensus.cc:388] T 00000000000000000000000000000000 P 964c81b7db814657a0df6455f493f626 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:46:47.394708 27476 raft_consensus.cc:482] T 00000000000000000000000000000000 P 964c81b7db814657a0df6455f493f626 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:46:47.394750 27476 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 964c81b7db814657a0df6455f493f626 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:46:47.395593 27476 raft_consensus.cc:504] T 00000000000000000000000000000000 P 964c81b7db814657a0df6455f493f626 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "964c81b7db814657a0df6455f493f626" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35711 } }
I20260928 14:46:47.395697 27476 leader_election.cc:302] T 00000000000000000000000000000000 P 964c81b7db814657a0df6455f493f626 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 964c81b7db814657a0df6455f493f626; no voters:
I20260928 14:46:47.395977 27476 leader_election.cc:288] T 00000000000000000000000000000000 P 964c81b7db814657a0df6455f493f626 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:46:47.396006 27481 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 964c81b7db814657a0df6455f493f626 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:46:47.396124 27481 raft_consensus.cc:686] T 00000000000000000000000000000000 P 964c81b7db814657a0df6455f493f626 [term 1 LEADER]: Becoming Leader. State: Replica: 964c81b7db814657a0df6455f493f626, State: Running, Role: LEADER
I20260928 14:46:47.396224 27481 consensus_queue.cc:237] T 00000000000000000000000000000000 P 964c81b7db814657a0df6455f493f626 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "964c81b7db814657a0df6455f493f626" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35711 } }
I20260928 14:46:47.396292 27476 sys_catalog.cc:564] T 00000000000000000000000000000000 P 964c81b7db814657a0df6455f493f626 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:46:47.396569 27482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 964c81b7db814657a0df6455f493f626 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "964c81b7db814657a0df6455f493f626" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "964c81b7db814657a0df6455f493f626" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35711 } } }
I20260928 14:46:47.396674 27482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 964c81b7db814657a0df6455f493f626 [sys.catalog]: This master's current role is: LEADER
I20260928 14:46:47.396867 27483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 964c81b7db814657a0df6455f493f626 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 964c81b7db814657a0df6455f493f626. Latest consensus state: current_term: 1 leader_uuid: "964c81b7db814657a0df6455f493f626" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "964c81b7db814657a0df6455f493f626" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35711 } } }
I20260928 14:46:47.396935 27483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 964c81b7db814657a0df6455f493f626 [sys.catalog]: This master's current role is: LEADER
Sep 28 14:46:47 dist-test-slave-pv0w krb5kdc[27395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606807, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
W20260928 14:46:47.398475 27496 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 964c81b7db814657a0df6455f493f626: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:46:47.398584 27496 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:46:47.398656 27486 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:46:47.398828 27486 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:46:47.400520 27486 catalog_manager.cc:1347] Generated new cluster ID: bbf57af977b341cf85499f5dd1291825
I20260928 14:46:47.400575 27486 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:46:47.401973 27479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:47.390427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36386 (local address 127.9.33.126:35711)
0928 14:46:47.390633 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:47.390649 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:47.390682 (+ 33us) server_negotiation.cc:407] Connection header received
0928 14:46:47.391210 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:47.391218 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:47.391273 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:47.391462 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:47.392265 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:47.393184 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:47.394020 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:47.394573 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:47.394683 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:47.397713 (+ 3030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:47.397718 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:47.397719 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:46:47.397734 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:47.400108 (+ 2374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:47.400593 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:47.400597 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:47.400598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:47.400709 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:47.400961 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:47.400962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:47.400962 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:47.401296 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:47.401471 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:47.401579 (+ 108us) server_negotiation.cc:299] Negotiation successful
0928 14:46:47.401737 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":93,"threads_started":1}
I20260928 14:46:47.411288 27486 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:46:47.412117 27486 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:46:47.416622 27486 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 964c81b7db814657a0df6455f493f626: Generated new TSK 0
I20260928 14:46:47.416836 27486 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.33.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab.
Entry for principal kudu/127.9.33.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab.
I20260928 14:46:47.451493 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.65:0
--local_ip_for_outbound_sockets=127.9.33.65
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:35711
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:40423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:46:47.535190 27502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:46:47.535413 27502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:46:47.535457 27502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:46:47.536731 27502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:46:47.536778 27502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:46:47.536806 27502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.65
I20260928 14:46:47.538295 27502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:40423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:35711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.65
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:46:47.538564 27502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:46:47.538813 27502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:46:47.544293 27508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:47.544361 27507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:47.544445 27510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:47.544348 27502 server_base.cc:1034] running on GCE node
I20260928 14:46:47.544742 27502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:46:47.545015 27502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:46:47.546195 27502 hybrid_clock.cc:648] HybridClock initialized: now 1790606807546179 us; error 29 us; skew 500 ppm
Sep 28 14:46:47 dist-test-slave-pv0w krb5kdc[27395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606807, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:46:47.548951 27502 init.cc:377] Logged in from keytab as kudu/127.9.33.65@KRBTEST.COM (short username kudu)
I20260928 14:46:47.549479 27502 webserver.cc:460] Webserver started at http://127.9.33.65:33805/ using document root <none> and password file <none>
I20260928 14:46:47.549757 27502 fs_manager.cc:359] Metadata directory not provided
I20260928 14:46:47.549854 27502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:46:47.550022 27502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:46:47.551288 27502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-0/data/instance:
uuid: "34c963e2f6984783a629051ef3779ee5"
format_stamp: "Formatted at 2026-09-28 14:46:47 on dist-test-slave-pv0w"
server_key: "7a36a7a0cf68ba9444bb123972847e5f"
server_key_iv: "84d89378ce9b27930c73fa22c2300297"
server_key_version: "encryptionkey@0"
I20260928 14:46:47.551633 27502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance:
uuid: "34c963e2f6984783a629051ef3779ee5"
format_stamp: "Formatted at 2026-09-28 14:46:47 on dist-test-slave-pv0w"
server_key: "7a36a7a0cf68ba9444bb123972847e5f"
server_key_iv: "84d89378ce9b27930c73fa22c2300297"
server_key_version: "encryptionkey@0"
I20260928 14:46:47.553160 27502 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:46:47.554062 27517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:47.554294 27502 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:46:47.554382 27502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-0/wal
uuid: "34c963e2f6984783a629051ef3779ee5"
format_stamp: "Formatted at 2026-09-28 14:46:47 on dist-test-slave-pv0w"
server_key: "7a36a7a0cf68ba9444bb123972847e5f"
server_key_iv: "84d89378ce9b27930c73fa22c2300297"
server_key_version: "encryptionkey@0"
I20260928 14:46:47.554443 27502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:46:47.567703 27502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:46:47.569800 27502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:46:47.569922 27502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:46:47.570153 27502 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:46:47.570434 27502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:46:47.570466 27502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:47.570490 27502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:46:47.570502 27502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:47.576789 27502 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.65:33585
I20260928 14:46:47.576821 27630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.65:33585 every 8 connection(s)
I20260928 14:46:47.577206 27502 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
Sep 28 14:46:47 dist-test-slave-pv0w krb5kdc[27395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606807, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:46:47.587016 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 27502
I20260928 14:46:47.587066 27479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:47.578009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:46175 (local address 127.9.33.126:35711)
0928 14:46:47.578077 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:47.578079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:47.578314 (+ 235us) server_negotiation.cc:407] Connection header received
0928 14:46:47.578875 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:47.578877 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:47.578889 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:47.578940 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:47.580014 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:47.580549 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:47.581592 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:47.581810 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:47.581896 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:47.584314 (+ 2418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:47.584316 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:47.584316 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:46:47.584324 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:47.585692 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:47.586105 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:47.586106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:47.586106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:47.586162 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:47.586385 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:47.586386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:47.586386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:47.586493 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:47.586542 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:47.586858 (+ 316us) server_negotiation.cc:299] Negotiation successful
0928 14:46:47.586953 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 14:46:47.587136 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance
I20260928 14:46:47.587172 27633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:47.577999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:35711 (local address 127.9.33.65:46175)
0928 14:46:47.578188 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:47.578224 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:47.578759 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:47.579014 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:47.579018 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:47.579628 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:46:47.579903 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:47.579909 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:47.580654 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:47.580655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:47.581483 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:47.581485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:47.581910 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:47.581911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:47.582046 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:47.582577 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:46:47.582586 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:46:47.584204 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:46:47.585778 (+ 1574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:46:47.585785 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:46:47.585787 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:46:47.586031 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:46:47.586221 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:46:47.586221 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:46:47.586222 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:46:47.586330 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:46:47.586582 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:46:47.586587 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:46:47.586720 (+ 133us) client_negotiation.cc:769] Sending connection context
0928 14:46:47.586808 (+ 88us) client_negotiation.cc:240] Negotiation successful
0928 14:46:47.586944 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":93,"threads_started":1}
I20260928 14:46:47.587399 9349 external_mini_cluster.cc:1428] Setting key 501c8d8ae54290be6e91381358ae5475
I20260928 14:46:47.587801 27631 heartbeater.cc:347] Connected to a master server at 127.9.33.126:35711
I20260928 14:46:47.587934 27631 heartbeater.cc:464] Registering TS with master...
I20260928 14:46:47.588169 27631 heartbeater.cc:511] Master 127.9.33.126:35711 requested a full tablet report, sending...
I20260928 14:46:47.588738 27440 ts_manager.cc:194] Registered new tserver with Master: 34c963e2f6984783a629051ef3779ee5 (127.9.33.65:33585)
I20260928 14:46:47.589357 27440 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.65@KRBTEST.COM'} at 127.9.33.65:46175
WARNING: no policy specified for kudu/127.9.33.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab.
Entry for principal kudu/127.9.33.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab.
I20260928 14:46:47.615584 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.66:0
--local_ip_for_outbound_sockets=127.9.33.66
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:35711
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:40423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:46:47.700151 27636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:46:47.700379 27636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:46:47.700405 27636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:46:47.701560 27636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:46:47.701597 27636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:46:47.701623 27636 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.66
I20260928 14:46:47.702998 27636 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:40423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:35711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.66
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:46:47.703189 27636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:46:47.703367 27636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:46:47.708433 27641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:47.708428 27644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:47.708428 27642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:47.708741 27636 server_base.cc:1034] running on GCE node
I20260928 14:46:47.708962 27636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:46:47.709209 27636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:46:47.710356 27636 hybrid_clock.cc:648] HybridClock initialized: now 1790606807710346 us; error 30 us; skew 500 ppm
Sep 28 14:46:47 dist-test-slave-pv0w krb5kdc[27395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606807, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:46:47.713013 27636 init.cc:377] Logged in from keytab as kudu/127.9.33.66@KRBTEST.COM (short username kudu)
I20260928 14:46:47.713505 27636 webserver.cc:460] Webserver started at http://127.9.33.66:44237/ using document root <none> and password file <none>
I20260928 14:46:47.713750 27636 fs_manager.cc:359] Metadata directory not provided
I20260928 14:46:47.713798 27636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:46:47.713936 27636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:46:47.715026 27636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-1/data/instance:
uuid: "aabec882a29e49f1bf21ee6c37a542a0"
format_stamp: "Formatted at 2026-09-28 14:46:47 on dist-test-slave-pv0w"
server_key: "72af8505b58ac83775e7f21f0d45b5df"
server_key_iv: "8efbd160ba6ee4b14eb204178b78674b"
server_key_version: "encryptionkey@0"
I20260928 14:46:47.715400 27636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance:
uuid: "aabec882a29e49f1bf21ee6c37a542a0"
format_stamp: "Formatted at 2026-09-28 14:46:47 on dist-test-slave-pv0w"
server_key: "72af8505b58ac83775e7f21f0d45b5df"
server_key_iv: "8efbd160ba6ee4b14eb204178b78674b"
server_key_version: "encryptionkey@0"
I20260928 14:46:47.716941 27636 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:46:47.717846 27651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:47.718087 27636 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:46:47.718169 27636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-1/wal
uuid: "aabec882a29e49f1bf21ee6c37a542a0"
format_stamp: "Formatted at 2026-09-28 14:46:47 on dist-test-slave-pv0w"
server_key: "72af8505b58ac83775e7f21f0d45b5df"
server_key_iv: "8efbd160ba6ee4b14eb204178b78674b"
server_key_version: "encryptionkey@0"
I20260928 14:46:47.718271 27636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:46:47.735405 27636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:46:47.737978 27636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:46:47.738104 27636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:46:47.738312 27636 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:46:47.738657 27636 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:46:47.738693 27636 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:47.738742 27636 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:46:47.738770 27636 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:47.744982 27636 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.66:35567
I20260928 14:46:47.745102 27764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.66:35567 every 8 connection(s)
I20260928 14:46:47.745474 27636 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
I20260928 14:46:47.751121 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 27636
I20260928 14:46:47.751255 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance
I20260928 14:46:47.751469 9349 external_mini_cluster.cc:1428] Setting key 5885af2f9fa0e21d5fcdd835276f9ff5
Sep 28 14:46:47 dist-test-slave-pv0w krb5kdc[27395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606807, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:46:47.757346 27479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:47.746319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:39015 (local address 127.9.33.126:35711)
0928 14:46:47.746394 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:47.746397 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:47.746724 (+ 327us) server_negotiation.cc:407] Connection header received
0928 14:46:47.747490 (+ 766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:47.747492 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:47.747506 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:47.747564 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:47.748902 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:47.749428 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:47.750414 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:47.750629 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:47.750691 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:47.753935 (+ 3244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:47.753936 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:47.753937 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:46:47.753954 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:47.755622 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:47.756097 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:47.756099 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:47.756099 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:47.756173 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:47.756501 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:47.756502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:47.756503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:47.756694 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:47.756770 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:47.757096 (+ 326us) server_negotiation.cc:299] Negotiation successful
0928 14:46:47.757205 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260928 14:46:47.757427 27767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:47.746301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:35711 (local address 127.9.33.66:39015)
0928 14:46:47.746589 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:47.746638 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:47.747365 (+ 727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:47.747661 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:47.747665 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:47.748531 (+ 866us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:46:47.748801 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:47.748807 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:47.749517 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:47.749518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:47.750325 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:47.750327 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:47.750733 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:47.750735 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:47.750880 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:47.751824 (+ 944us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:46:47.751833 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:46:47.753795 (+ 1962us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:46:47.755712 (+ 1917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:46:47.755767 (+ 55us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:46:47.755770 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:46:47.756018 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:46:47.756276 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:46:47.756277 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:46:47.756277 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:46:47.756406 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:46:47.756772 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:46:47.756779 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:46:47.756924 (+ 145us) client_negotiation.cc:769] Sending connection context
0928 14:46:47.757023 (+ 99us) client_negotiation.cc:240] Negotiation successful
0928 14:46:47.757217 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":91,"threads_started":1}
I20260928 14:46:47.757957 27765 heartbeater.cc:347] Connected to a master server at 127.9.33.126:35711
I20260928 14:46:47.758077 27765 heartbeater.cc:464] Registering TS with master...
I20260928 14:46:47.758302 27765 heartbeater.cc:511] Master 127.9.33.126:35711 requested a full tablet report, sending...
I20260928 14:46:47.758790 27440 ts_manager.cc:194] Registered new tserver with Master: aabec882a29e49f1bf21ee6c37a542a0 (127.9.33.66:35567)
I20260928 14:46:47.759243 27440 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.66@KRBTEST.COM'} at 127.9.33.66:39015
WARNING: no policy specified for kudu/127.9.33.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab.
Entry for principal kudu/127.9.33.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab.
I20260928 14:46:47.784137 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.67:0
--local_ip_for_outbound_sockets=127.9.33.67
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:35711
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:40423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:46:47.867307 27770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:46:47.867537 27770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:46:47.867563 27770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:46:47.868866 27770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:46:47.868917 27770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:46:47.868947 27770 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.67
I20260928 14:46:47.870483 27770 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:40423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:35711
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.67
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:46:47.870755 27770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:46:47.871014 27770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:46:47.876483 27778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:47.876487 27776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:47.876477 27775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:47.876777 27770 server_base.cc:1034] running on GCE node
I20260928 14:46:47.877031 27770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:46:47.877317 27770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:46:47.878551 27770 hybrid_clock.cc:648] HybridClock initialized: now 1790606807878499 us; error 76 us; skew 500 ppm
Sep 28 14:46:47 dist-test-slave-pv0w krb5kdc[27395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606807, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:46:47.881321 27770 init.cc:377] Logged in from keytab as kudu/127.9.33.67@KRBTEST.COM (short username kudu)
I20260928 14:46:47.881886 27770 webserver.cc:460] Webserver started at http://127.9.33.67:34789/ using document root <none> and password file <none>
I20260928 14:46:47.882143 27770 fs_manager.cc:359] Metadata directory not provided
I20260928 14:46:47.882193 27770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:46:47.882364 27770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:46:47.883517 27770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-2/data/instance:
uuid: "9972e26a1a704cfea88427336b5d1610"
format_stamp: "Formatted at 2026-09-28 14:46:47 on dist-test-slave-pv0w"
server_key: "15af9ccc09080f7cef78f1d20edcd5a7"
server_key_iv: "6784ba9d92e864ae3d32c2a88a566b1d"
server_key_version: "encryptionkey@0"
I20260928 14:46:47.883944 27770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance:
uuid: "9972e26a1a704cfea88427336b5d1610"
format_stamp: "Formatted at 2026-09-28 14:46:47 on dist-test-slave-pv0w"
server_key: "15af9ccc09080f7cef78f1d20edcd5a7"
server_key_iv: "6784ba9d92e864ae3d32c2a88a566b1d"
server_key_version: "encryptionkey@0"
I20260928 14:46:47.885345 27770 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:46:47.886224 27785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:47.886438 27770 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:46:47.886512 27770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-2/wal
uuid: "9972e26a1a704cfea88427336b5d1610"
format_stamp: "Formatted at 2026-09-28 14:46:47 on dist-test-slave-pv0w"
server_key: "15af9ccc09080f7cef78f1d20edcd5a7"
server_key_iv: "6784ba9d92e864ae3d32c2a88a566b1d"
server_key_version: "encryptionkey@0"
I20260928 14:46:47.886608 27770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:46:47.900442 27770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:46:47.902745 27770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:46:47.902901 27770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:46:47.903172 27770 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:46:47.903512 27770 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:46:47.903545 27770 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:47.903569 27770 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:46:47.903628 27770 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:47.909886 27770 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.67:38511
I20260928 14:46:47.909930 27898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.67:38511 every 8 connection(s)
I20260928 14:46:47.910256 27770 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
Sep 28 14:46:47 dist-test-slave-pv0w krb5kdc[27395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606807, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:46:47.919270 27479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:47.910987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:44199 (local address 127.9.33.126:35711)
0928 14:46:47.911070 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:47.911073 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:47.911233 (+ 160us) server_negotiation.cc:407] Connection header received
0928 14:46:47.911764 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:47.911766 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:47.911786 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:47.911840 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:47.912776 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:47.913272 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:47.914284 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:47.914472 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:47.914529 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:47.916749 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:47.916750 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:47.916750 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:46:47.916757 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:47.918061 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:47.918439 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:47.918439 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:47.918440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:47.918490 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:47.918738 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:47.918739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:47.918739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:47.918841 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:47.918893 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:47.919137 (+ 244us) server_negotiation.cc:299] Negotiation successful
0928 14:46:47.919202 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260928 14:46:47.919446 27901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:47.910942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:35711 (local address 127.9.33.67:44199)
0928 14:46:47.911112 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:47.911150 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:47.911628 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:47.911902 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:47.911906 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:47.912476 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:46:47.912691 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:47.912696 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:47.913349 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:47.913350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:47.914200 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:47.914203 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:47.914549 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:47.914550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:47.914675 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:47.915165 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:46:47.915172 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:46:47.916638 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:46:47.918145 (+ 1507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:46:47.918151 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:46:47.918153 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:46:47.918372 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:46:47.918548 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:46:47.918549 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:46:47.918549 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:46:47.918680 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:46:47.918898 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:46:47.918903 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:46:47.919023 (+ 120us) client_negotiation.cc:769] Sending connection context
0928 14:46:47.919106 (+ 83us) client_negotiation.cc:240] Negotiation successful
0928 14:46:47.919236 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":92,"threads_started":1}
I20260928 14:46:47.919477 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 27770
I20260928 14:46:47.919549 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance
I20260928 14:46:47.919690 9349 external_mini_cluster.cc:1428] Setting key 3f85b6e623222556c552dbf824f6ff8d
I20260928 14:46:47.919987 27899 heartbeater.cc:347] Connected to a master server at 127.9.33.126:35711
I20260928 14:46:47.920077 27899 heartbeater.cc:464] Registering TS with master...
I20260928 14:46:47.920277 27899 heartbeater.cc:511] Master 127.9.33.126:35711 requested a full tablet report, sending...
I20260928 14:46:47.920723 27440 ts_manager.cc:194] Registered new tserver with Master: 9972e26a1a704cfea88427336b5d1610 (127.9.33.67:38511)
I20260928 14:46:47.921150 27440 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.67@KRBTEST.COM'} at 127.9.33.67:44199
I20260928 14:46:47.931676 9349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:46:47.940125 27479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:47.933183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36392 (local address 127.9.33.126:35711)
0928 14:46:47.933298 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:47.933302 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:47.933423 (+ 121us) server_negotiation.cc:407] Connection header received
0928 14:46:47.933511 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:47.933512 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:47.933522 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:47.933573 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:47.934195 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:47.934760 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:47.935579 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:47.935875 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:47.935970 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:47.937351 (+ 1381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:47.937353 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:47.937353 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:46:47.937362 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:47.938700 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:47.939170 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:47.939172 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:47.939172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:47.939232 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:47.939465 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:47.939467 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:47.939467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:47.939671 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:47.939824 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:47.939948 (+ 124us) server_negotiation.cc:299] Negotiation successful
0928 14:46:47.940018 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260928 14:46:47.941810 27440 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36392:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:46:47.943452 27440 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:46:47.950765 27915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:47.946522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41304 (local address 127.9.33.65:33585)
0928 14:46:47.946731 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:47.946736 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:47.946876 (+ 140us) server_negotiation.cc:407] Connection header received
0928 14:46:47.946928 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:47.946930 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:47.946946 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:47.947192 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:46:47.947270 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:47.948409 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:47.949702 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:47.950294 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:47.950388 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:47.950455 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:47.950615 (+ 160us) server_negotiation.cc:299] Negotiation successful
0928 14:46:47.950658 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":86,"threads_started":1}
I20260928 14:46:47.950788 27913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:47.946146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:33585 (local address 127.0.0.1:41304)
0928 14:46:47.946785 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:47.946795 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:47.946863 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:47.947049 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:47.947049 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:47.947052 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:46:47.947111 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:47.947114 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:47.948523 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:47.948524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:47.949617 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:47.949619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:47.950388 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:47.950389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:47.950477 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:47.950493 (+ 16us) client_negotiation.cc:769] Sending connection context
0928 14:46:47.950584 (+ 91us) client_negotiation.cc:240] Negotiation successful
0928 14:46:47.950658 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"mutex_wait_us":27,"spinlock_wait_cycles":384,"thread_start_us":81,"threads_started":1}
I20260928 14:46:47.950793 27912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:47.946278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40362 (local address 127.9.33.66:35567)
0928 14:46:47.946448 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:47.946454 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:47.946608 (+ 154us) server_negotiation.cc:407] Connection header received
0928 14:46:47.946782 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:47.946785 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:47.946830 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:47.946911 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:46:47.947201 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:47.948074 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:47.949552 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:47.950317 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:47.950393 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:47.950454 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:47.950670 (+ 216us) server_negotiation.cc:299] Negotiation successful
0928 14:46:47.950712 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":101,"threads_started":1}
I20260928 14:46:47.950790 27911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:47.946359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:38511 (local address 127.0.0.1:45430)
0928 14:46:47.946941 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:47.946950 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:47.946990 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:47.947273 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:47.947274 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:47.947276 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:46:47.947308 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:47.947309 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:47.948573 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:47.948574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:47.949559 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:47.949562 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:47.950494 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:47.950495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:47.950566 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:47.950573 (+ 7us) client_negotiation.cc:769] Sending connection context
0928 14:46:47.950625 (+ 52us) client_negotiation.cc:240] Negotiation successful
0928 14:46:47.950689 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"mutex_wait_us":2,"thread_start_us":73,"threads_started":1}
I20260928 14:46:47.951162 27910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:47.946146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:35567 (local address 127.0.0.1:40362)
0928 14:46:47.946482 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:47.946522 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:47.946626 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:47.947021 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:47.947024 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:47.947027 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:46:47.947106 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:47.947109 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:47.948185 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:47.948186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:47.949454 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:47.949457 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:47.950492 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:47.950494 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:47.950566 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:47.950575 (+ 9us) client_negotiation.cc:769] Sending connection context
0928 14:46:47.950948 (+ 373us) client_negotiation.cc:240] Negotiation successful
0928 14:46:47.951007 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"mutex_wait_us":3,"thread_start_us":72,"threads_started":1}
I20260928 14:46:47.951664 27565 tablet_service.cc:1467] Processing CreateTablet for tablet c2d01c594dd34ad58c5dc6ecfeb7ba41 (DEFAULT_TABLE table=test-table [id=4a63caf898144532af723a9528a27fb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:46:47.951982 27699 tablet_service.cc:1467] Processing CreateTablet for tablet c2d01c594dd34ad58c5dc6ecfeb7ba41 (DEFAULT_TABLE table=test-table [id=4a63caf898144532af723a9528a27fb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:46:47.952096 27565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2d01c594dd34ad58c5dc6ecfeb7ba41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:46:47.952297 27699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2d01c594dd34ad58c5dc6ecfeb7ba41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:46:47.954945 27916 tablet_bootstrap.cc:492] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P aabec882a29e49f1bf21ee6c37a542a0: Bootstrap starting.
I20260928 14:46:47.955166 27917 tablet_bootstrap.cc:492] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 34c963e2f6984783a629051ef3779ee5: Bootstrap starting.
I20260928 14:46:47.955374 27914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:47.946486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45430 (local address 127.9.33.67:38511)
0928 14:46:47.946713 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:47.946718 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:47.947045 (+ 327us) server_negotiation.cc:407] Connection header received
0928 14:46:47.947089 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:47.947091 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:47.947105 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:47.947191 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:46:47.947392 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:47.948487 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:47.949665 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:47.950328 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:47.950394 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:47.950454 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:47.950827 (+ 373us) server_negotiation.cc:299] Negotiation successful
0928 14:46:47.955256 (+ 4429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":110,"threads_started":1}
I20260928 14:46:47.955861 27916 tablet_bootstrap.cc:654] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P aabec882a29e49f1bf21ee6c37a542a0: Neither blocks nor log segments found. Creating new log.
I20260928 14:46:47.955926 27917 tablet_bootstrap.cc:654] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 34c963e2f6984783a629051ef3779ee5: Neither blocks nor log segments found. Creating new log.
I20260928 14:46:47.956208 27833 tablet_service.cc:1467] Processing CreateTablet for tablet c2d01c594dd34ad58c5dc6ecfeb7ba41 (DEFAULT_TABLE table=test-table [id=4a63caf898144532af723a9528a27fb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:46:47.956440 27917 log.cc:824] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 34c963e2f6984783a629051ef3779ee5: Log is configured to *not* fsync() on all Append() calls
I20260928 14:46:47.956454 27916 log.cc:824] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P aabec882a29e49f1bf21ee6c37a542a0: Log is configured to *not* fsync() on all Append() calls
I20260928 14:46:47.956570 27833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2d01c594dd34ad58c5dc6ecfeb7ba41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:46:47.957321 27916 tablet_bootstrap.cc:492] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P aabec882a29e49f1bf21ee6c37a542a0: No bootstrap required, opened a new log
I20260928 14:46:47.957402 27916 ts_tablet_manager.cc:1397] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P aabec882a29e49f1bf21ee6c37a542a0: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260928 14:46:47.959154 27916 raft_consensus.cc:348] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P aabec882a29e49f1bf21ee6c37a542a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34c963e2f6984783a629051ef3779ee5" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33585 } } peers { permanent_uuid: "aabec882a29e49f1bf21ee6c37a542a0" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 35567 } } peers { permanent_uuid: "9972e26a1a704cfea88427336b5d1610" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 38511 } }
I20260928 14:46:47.959326 27916 raft_consensus.cc:374] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P aabec882a29e49f1bf21ee6c37a542a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:46:47.959370 27916 raft_consensus.cc:729] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P aabec882a29e49f1bf21ee6c37a542a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aabec882a29e49f1bf21ee6c37a542a0, State: Initialized, Role: FOLLOWER
I20260928 14:46:47.959395 27920 tablet_bootstrap.cc:492] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 9972e26a1a704cfea88427336b5d1610: Bootstrap starting.
I20260928 14:46:47.959482 27916 consensus_queue.cc:260] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P aabec882a29e49f1bf21ee6c37a542a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34c963e2f6984783a629051ef3779ee5" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33585 } } peers { permanent_uuid: "aabec882a29e49f1bf21ee6c37a542a0" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 35567 } } peers { permanent_uuid: "9972e26a1a704cfea88427336b5d1610" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 38511 } }
I20260928 14:46:47.959800 27916 ts_tablet_manager.cc:1428] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P aabec882a29e49f1bf21ee6c37a542a0: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260928 14:46:47.959925 27765 heartbeater.cc:503] Master 127.9.33.126:35711 was elected leader, sending a full tablet report...
I20260928 14:46:47.960037 27920 tablet_bootstrap.cc:654] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 9972e26a1a704cfea88427336b5d1610: Neither blocks nor log segments found. Creating new log.
I20260928 14:46:47.960474 27917 tablet_bootstrap.cc:492] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 34c963e2f6984783a629051ef3779ee5: No bootstrap required, opened a new log
I20260928 14:46:47.960559 27917 ts_tablet_manager.cc:1397] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 34c963e2f6984783a629051ef3779ee5: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260928 14:46:47.960605 27920 log.cc:824] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 9972e26a1a704cfea88427336b5d1610: Log is configured to *not* fsync() on all Append() calls
I20260928 14:46:47.961279 27920 tablet_bootstrap.cc:492] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 9972e26a1a704cfea88427336b5d1610: No bootstrap required, opened a new log
I20260928 14:46:47.961330 27920 ts_tablet_manager.cc:1397] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 9972e26a1a704cfea88427336b5d1610: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260928 14:46:47.962170 27917 raft_consensus.cc:348] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 34c963e2f6984783a629051ef3779ee5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34c963e2f6984783a629051ef3779ee5" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33585 } } peers { permanent_uuid: "aabec882a29e49f1bf21ee6c37a542a0" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 35567 } } peers { permanent_uuid: "9972e26a1a704cfea88427336b5d1610" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 38511 } }
I20260928 14:46:47.962332 27917 raft_consensus.cc:374] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 34c963e2f6984783a629051ef3779ee5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:46:47.962376 27917 raft_consensus.cc:729] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 34c963e2f6984783a629051ef3779ee5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34c963e2f6984783a629051ef3779ee5, State: Initialized, Role: FOLLOWER
I20260928 14:46:47.962522 27917 consensus_queue.cc:260] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 34c963e2f6984783a629051ef3779ee5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34c963e2f6984783a629051ef3779ee5" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33585 } } peers { permanent_uuid: "aabec882a29e49f1bf21ee6c37a542a0" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 35567 } } peers { permanent_uuid: "9972e26a1a704cfea88427336b5d1610" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 38511 } }
I20260928 14:46:47.962801 27917 ts_tablet_manager.cc:1428] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 34c963e2f6984783a629051ef3779ee5: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260928 14:46:47.962862 27920 raft_consensus.cc:348] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 9972e26a1a704cfea88427336b5d1610 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34c963e2f6984783a629051ef3779ee5" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33585 } } peers { permanent_uuid: "aabec882a29e49f1bf21ee6c37a542a0" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 35567 } } peers { permanent_uuid: "9972e26a1a704cfea88427336b5d1610" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 38511 } }
I20260928 14:46:47.962982 27631 heartbeater.cc:503] Master 127.9.33.126:35711 was elected leader, sending a full tablet report...
I20260928 14:46:47.963035 27920 raft_consensus.cc:374] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 9972e26a1a704cfea88427336b5d1610 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:46:47.963073 27920 raft_consensus.cc:729] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 9972e26a1a704cfea88427336b5d1610 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9972e26a1a704cfea88427336b5d1610, State: Initialized, Role: FOLLOWER
I20260928 14:46:47.963188 27920 consensus_queue.cc:260] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 9972e26a1a704cfea88427336b5d1610 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34c963e2f6984783a629051ef3779ee5" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33585 } } peers { permanent_uuid: "aabec882a29e49f1bf21ee6c37a542a0" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 35567 } } peers { permanent_uuid: "9972e26a1a704cfea88427336b5d1610" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 38511 } }
I20260928 14:46:47.963482 27920 ts_tablet_manager.cc:1428] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 9972e26a1a704cfea88427336b5d1610: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260928 14:46:47.963516 27899 heartbeater.cc:503] Master 127.9.33.126:35711 was elected leader, sending a full tablet report...
W20260928 14:46:47.996783 27766 tablet.cc:2369] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P aabec882a29e49f1bf21ee6c37a542a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:46:48.078161 27632 tablet.cc:2369] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 34c963e2f6984783a629051ef3779ee5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260928 14:46:48.160825 27900 tablet.cc:2369] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 9972e26a1a704cfea88427336b5d1610: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:46:48.356118 27921 raft_consensus.cc:482] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P aabec882a29e49f1bf21ee6c37a542a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:46:48.356244 27921 raft_consensus.cc:504] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P aabec882a29e49f1bf21ee6c37a542a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34c963e2f6984783a629051ef3779ee5" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33585 } } peers { permanent_uuid: "aabec882a29e49f1bf21ee6c37a542a0" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 35567 } } peers { permanent_uuid: "9972e26a1a704cfea88427336b5d1610" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 38511 } }
I20260928 14:46:48.356599 27921 leader_election.cc:288] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P aabec882a29e49f1bf21ee6c37a542a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34c963e2f6984783a629051ef3779ee5 (127.9.33.65:33585), 9972e26a1a704cfea88427336b5d1610 (127.9.33.67:38511)
I20260928 14:46:48.359946 27915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:48.356900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:49269 (local address 127.9.33.65:33585)
0928 14:46:48.356952 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:48.356954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:48.357054 (+ 100us) server_negotiation.cc:407] Connection header received
0928 14:46:48.357224 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:48.357226 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:48.357242 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:48.357323 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:46:48.357508 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:48.358049 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:48.359166 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:48.359545 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:48.359623 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:48.359635 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:48.359813 (+ 178us) server_negotiation.cc:299] Negotiation successful
0928 14:46:48.359862 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 14:46:48.359961 27914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:48.356963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:35767 (local address 127.9.33.67:38511)
0928 14:46:48.357045 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:48.357047 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:48.357058 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:46:48.357136 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:48.357138 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:48.357148 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:48.357210 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:46:48.357376 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:48.357870 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:48.359109 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:48.359513 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:48.359603 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:48.359615 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:48.359828 (+ 213us) server_negotiation.cc:299] Negotiation successful
0928 14:46:48.359873 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 14:46:48.360021 27925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:48.356841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:38511 (local address 127.9.33.66:35767)
0928 14:46:48.356996 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:48.357006 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:48.357068 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:48.357218 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:48.357219 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:48.357221 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:46:48.357301 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:48.357303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:48.357968 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:48.357969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:48.358999 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:48.359002 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:48.359606 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:48.359608 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:48.359689 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:48.359700 (+ 11us) client_negotiation.cc:769] Sending connection context
0928 14:46:48.359815 (+ 115us) client_negotiation.cc:240] Negotiation successful
0928 14:46:48.359862 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":65,"threads_started":1}
I20260928 14:46:48.360038 27926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:48.356799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:33585 (local address 127.9.33.66:49269)
0928 14:46:48.356967 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:48.356980 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:48.357057 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:48.357330 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:48.357331 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:48.357333 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:46:48.357404 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:48.357410 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:48.358145 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:48.358146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:48.359067 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:48.359070 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:48.359638 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:48.359639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:48.359714 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:48.359735 (+ 21us) client_negotiation.cc:769] Sending connection context
0928 14:46:48.359805 (+ 70us) client_negotiation.cc:240] Negotiation successful
0928 14:46:48.359854 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":84,"threads_started":1}
I20260928 14:46:48.360530 27853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2d01c594dd34ad58c5dc6ecfeb7ba41" candidate_uuid: "aabec882a29e49f1bf21ee6c37a542a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9972e26a1a704cfea88427336b5d1610" is_pre_election: true
I20260928 14:46:48.360532 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2d01c594dd34ad58c5dc6ecfeb7ba41" candidate_uuid: "aabec882a29e49f1bf21ee6c37a542a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34c963e2f6984783a629051ef3779ee5" is_pre_election: true
I20260928 14:46:48.360711 27585 raft_consensus.cc:2454] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 34c963e2f6984783a629051ef3779ee5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aabec882a29e49f1bf21ee6c37a542a0 in term 0.
I20260928 14:46:48.360711 27853 raft_consensus.cc:2454] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 9972e26a1a704cfea88427336b5d1610 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aabec882a29e49f1bf21ee6c37a542a0 in term 0.
I20260928 14:46:48.360975 27655 leader_election.cc:302] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P aabec882a29e49f1bf21ee6c37a542a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34c963e2f6984783a629051ef3779ee5, aabec882a29e49f1bf21ee6c37a542a0; no voters:
I20260928 14:46:48.361138 27921 raft_consensus.cc:2781] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P aabec882a29e49f1bf21ee6c37a542a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:46:48.361203 27921 raft_consensus.cc:482] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P aabec882a29e49f1bf21ee6c37a542a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:46:48.361220 27921 raft_consensus.cc:3037] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P aabec882a29e49f1bf21ee6c37a542a0 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:46:48.361948 27921 raft_consensus.cc:504] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P aabec882a29e49f1bf21ee6c37a542a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34c963e2f6984783a629051ef3779ee5" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33585 } } peers { permanent_uuid: "aabec882a29e49f1bf21ee6c37a542a0" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 35567 } } peers { permanent_uuid: "9972e26a1a704cfea88427336b5d1610" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 38511 } }
I20260928 14:46:48.362103 27921 leader_election.cc:288] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P aabec882a29e49f1bf21ee6c37a542a0 [CANDIDATE]: Term 1 election: Requested vote from peers 34c963e2f6984783a629051ef3779ee5 (127.9.33.65:33585), 9972e26a1a704cfea88427336b5d1610 (127.9.33.67:38511)
I20260928 14:46:48.362290 27585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2d01c594dd34ad58c5dc6ecfeb7ba41" candidate_uuid: "aabec882a29e49f1bf21ee6c37a542a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34c963e2f6984783a629051ef3779ee5"
I20260928 14:46:48.362329 27853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2d01c594dd34ad58c5dc6ecfeb7ba41" candidate_uuid: "aabec882a29e49f1bf21ee6c37a542a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9972e26a1a704cfea88427336b5d1610"
I20260928 14:46:48.362385 27585 raft_consensus.cc:3037] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 34c963e2f6984783a629051ef3779ee5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:46:48.362437 27853 raft_consensus.cc:3037] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 9972e26a1a704cfea88427336b5d1610 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:46:48.363212 27853 raft_consensus.cc:2454] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 9972e26a1a704cfea88427336b5d1610 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aabec882a29e49f1bf21ee6c37a542a0 in term 1.
I20260928 14:46:48.363221 27585 raft_consensus.cc:2454] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 34c963e2f6984783a629051ef3779ee5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aabec882a29e49f1bf21ee6c37a542a0 in term 1.
I20260928 14:46:48.363420 27653 leader_election.cc:302] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P aabec882a29e49f1bf21ee6c37a542a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9972e26a1a704cfea88427336b5d1610, aabec882a29e49f1bf21ee6c37a542a0; no voters:
I20260928 14:46:48.363523 27921 raft_consensus.cc:2781] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P aabec882a29e49f1bf21ee6c37a542a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:46:48.363667 27921 raft_consensus.cc:686] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P aabec882a29e49f1bf21ee6c37a542a0 [term 1 LEADER]: Becoming Leader. State: Replica: aabec882a29e49f1bf21ee6c37a542a0, State: Running, Role: LEADER
I20260928 14:46:48.363835 27921 consensus_queue.cc:237] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P aabec882a29e49f1bf21ee6c37a542a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34c963e2f6984783a629051ef3779ee5" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33585 } } peers { permanent_uuid: "aabec882a29e49f1bf21ee6c37a542a0" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 35567 } } peers { permanent_uuid: "9972e26a1a704cfea88427336b5d1610" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 38511 } }
I20260928 14:46:48.364825 27440 catalog_manager.cc:5495] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P aabec882a29e49f1bf21ee6c37a542a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to aabec882a29e49f1bf21ee6c37a542a0 (127.9.33.66). New cstate: current_term: 1 leader_uuid: "aabec882a29e49f1bf21ee6c37a542a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34c963e2f6984783a629051ef3779ee5" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aabec882a29e49f1bf21ee6c37a542a0" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 35567 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9972e26a1a704cfea88427336b5d1610" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 38511 } health_report { overall_health: UNKNOWN } } }
I20260928 14:46:48.392961 27912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:48.390540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40378 (local address 127.9.33.66:35567)
0928 14:46:48.390674 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:48.390676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:48.390684 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:46:48.390735 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:48.390736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:48.390757 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:48.390817 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:46:48.390931 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:48.391413 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:48.392310 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:48.392494 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:48.392546 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:48.392684 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:46:48.392773 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:46:48.392842 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:48.392891 (+ 49us) server_negotiation.cc:299] Negotiation successful
0928 14:46:48.392899 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260928 14:46:48.394834 27853 raft_consensus.cc:1261] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 9972e26a1a704cfea88427336b5d1610 [term 1 FOLLOWER]: Refusing update from remote peer aabec882a29e49f1bf21ee6c37a542a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:46:48.394869 27585 raft_consensus.cc:1261] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 34c963e2f6984783a629051ef3779ee5 [term 1 FOLLOWER]: Refusing update from remote peer aabec882a29e49f1bf21ee6c37a542a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:46:48.395227 27927 consensus_queue.cc:1035] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P aabec882a29e49f1bf21ee6c37a542a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9972e26a1a704cfea88427336b5d1610" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 38511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:46:48.395334 27921 consensus_queue.cc:1035] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P aabec882a29e49f1bf21ee6c37a542a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34c963e2f6984783a629051ef3779ee5" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 33585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:46:48.398494 27930 mvcc.cc:205] Tried to move back new op lower bound from 7334325487182348288 to 7334325487058997248. Current Snapshot: MvccSnapshot[applied={T|T < 7334325487182348288 or (T in {7334325487182348288})}]
I20260928 14:46:48.398994 27931 mvcc.cc:205] Tried to move back new op lower bound from 7334325487182348288 to 7334325487058997248. Current Snapshot: MvccSnapshot[applied={T|T < 7334325487182348288}]
I20260928 14:46:48.400527 27439 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:46:48.400617 27439 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36392:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:46:48.401394 27439 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 964c81b7db814657a0df6455f493f626: Sending DeleteTablet for 3 replicas of tablet c2d01c594dd34ad58c5dc6ecfeb7ba41
I20260928 14:46:48.401695 27699 tablet_service.cc:1514] Processing DeleteTablet for tablet c2d01c594dd34ad58c5dc6ecfeb7ba41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:46:48 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:40362
I20260928 14:46:48.401762 27565 tablet_service.cc:1514] Processing DeleteTablet for tablet c2d01c594dd34ad58c5dc6ecfeb7ba41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:46:48 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:41304
I20260928 14:46:48.401762 27833 tablet_service.cc:1514] Processing DeleteTablet for tablet c2d01c594dd34ad58c5dc6ecfeb7ba41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:46:48 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:45430
I20260928 14:46:48.402051 27943 tablet_replica.cc:330] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 34c963e2f6984783a629051ef3779ee5: stopping tablet replica
I20260928 14:46:48.402133 27943 raft_consensus.cc:2229] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 34c963e2f6984783a629051ef3779ee5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:46:48.402207 27943 raft_consensus.cc:2258] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 34c963e2f6984783a629051ef3779ee5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:46:48.402320 27941 tablet_replica.cc:330] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P aabec882a29e49f1bf21ee6c37a542a0: stopping tablet replica
I20260928 14:46:48.402377 27941 raft_consensus.cc:2229] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P aabec882a29e49f1bf21ee6c37a542a0 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:46:48.402462 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 27502
I20260928 14:46:48.402494 27941 raft_consensus.cc:2258] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P aabec882a29e49f1bf21ee6c37a542a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:46:48.402639 27942 tablet_replica.cc:330] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 9972e26a1a704cfea88427336b5d1610: stopping tablet replica
I20260928 14:46:48.402722 27942 raft_consensus.cc:2229] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 9972e26a1a704cfea88427336b5d1610 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:46:48.402803 27942 raft_consensus.cc:2258] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 9972e26a1a704cfea88427336b5d1610 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:46:48.403256 27942 ts_tablet_manager.cc:1905] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 9972e26a1a704cfea88427336b5d1610: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:46:48.403277 27941 ts_tablet_manager.cc:1905] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P aabec882a29e49f1bf21ee6c37a542a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:46:48.405121 27942 ts_tablet_manager.cc:1918] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 9972e26a1a704cfea88427336b5d1610: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:46:48.405195 27942 log.cc:1196] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 9972e26a1a704cfea88427336b5d1610: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-2/wal/wals/c2d01c594dd34ad58c5dc6ecfeb7ba41
I20260928 14:46:48.405319 27941 ts_tablet_manager.cc:1918] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P aabec882a29e49f1bf21ee6c37a542a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:46:48.405390 27941 log.cc:1196] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P aabec882a29e49f1bf21ee6c37a542a0: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790606561545179-9349-0/minicluster-data/ts-1/wal/wals/c2d01c594dd34ad58c5dc6ecfeb7ba41
I20260928 14:46:48.405453 27942 ts_tablet_manager.cc:1939] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P 9972e26a1a704cfea88427336b5d1610: Deleting consensus metadata
I20260928 14:46:48.405661 27941 ts_tablet_manager.cc:1939] T c2d01c594dd34ad58c5dc6ecfeb7ba41 P aabec882a29e49f1bf21ee6c37a542a0: Deleting consensus metadata
I20260928 14:46:48.405908 27425 catalog_manager.cc:4841] TS 9972e26a1a704cfea88427336b5d1610 (127.9.33.67:38511): tablet c2d01c594dd34ad58c5dc6ecfeb7ba41 (table test-table [id=4a63caf898144532af723a9528a27fb8]) successfully deleted
I20260928 14:46:48.406569 27425 catalog_manager.cc:4841] TS aabec882a29e49f1bf21ee6c37a542a0 (127.9.33.66:35567): tablet c2d01c594dd34ad58c5dc6ecfeb7ba41 (table test-table [id=4a63caf898144532af723a9528a27fb8]) successfully deleted
W20260928 14:46:48.410104 27427 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.33.65:33585 (error 108)
I20260928 14:46:48.410725 27910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:48.410564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:33585 (local address 127.0.0.1:41310)
0928 14:46:48.410644 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:48.410689 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.65:33585: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260928 14:46:48.410758 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 27636
W20260928 14:46:48.411926 27427 catalog_manager.cc:4584] TS 34c963e2f6984783a629051ef3779ee5 (127.9.33.65:33585): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c2d01c594dd34ad58c5dc6ecfeb7ba41: Network error: Client connection negotiation failed: client connection to 127.9.33.65:33585: connect: Connection refused (error 111)
I20260928 14:46:48.416338 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 27770
I20260928 14:46:48.420993 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 27408
2026-09-28T14:46:48Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2325 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-28T14:46:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:46:48Z Disabled control of system clock
I20260928 14:46:48.450348 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:37565
--webserver_interface=127.9.33.126
--webserver_port=0
--builtin_ntp_servers=127.9.33.84:36863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:37565
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 14:46:48.532858 27948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:46:48.533105 27948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:46:48.533128 27948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:46:48.534369 27948 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:46:48.534418 27948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:46:48.534432 27948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:46:48.534444 27948 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:46:48.534462 27948 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:46:48.535959 27948 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:36863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:37565
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:37565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:46:48.536262 27948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:46:48.536602 27948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:46:48.543051 27954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:48.543135 27956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:48.543188 27953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:48.543291 27948 server_base.cc:1034] running on GCE node
I20260928 14:46:48.543552 27948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:46:48.543939 27948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:46:48.545127 27948 hybrid_clock.cc:648] HybridClock initialized: now 1790606808545118 us; error 37 us; skew 500 ppm
I20260928 14:46:48.551443 27948 webserver.cc:460] Webserver started at http://127.9.33.126:46347/ using document root <none> and password file <none>
I20260928 14:46:48.551836 27948 fs_manager.cc:359] Metadata directory not provided
I20260928 14:46:48.551920 27948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:46:48.552104 27948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:46:48.553318 27948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "27ec6806cbc84f44a4cda437105f45e5"
format_stamp: "Formatted at 2026-09-28 14:46:48 on dist-test-slave-pv0w"
server_key: "410e60a6cd6a4ea4323a6ac20b9e211a"
server_key_iv: "9f7ae5ae8843fcd2e67d7de7fed96c8f"
server_key_version: "encryptionkey@0"
I20260928 14:46:48.553714 27948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "27ec6806cbc84f44a4cda437105f45e5"
format_stamp: "Formatted at 2026-09-28 14:46:48 on dist-test-slave-pv0w"
server_key: "410e60a6cd6a4ea4323a6ac20b9e211a"
server_key_iv: "9f7ae5ae8843fcd2e67d7de7fed96c8f"
server_key_version: "encryptionkey@0"
I20260928 14:46:48.555228 27948 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:46:48.556121 27962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:48.556298 27948 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260928 14:46:48.556404 27948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "27ec6806cbc84f44a4cda437105f45e5"
format_stamp: "Formatted at 2026-09-28 14:46:48 on dist-test-slave-pv0w"
server_key: "410e60a6cd6a4ea4323a6ac20b9e211a"
server_key_iv: "9f7ae5ae8843fcd2e67d7de7fed96c8f"
server_key_version: "encryptionkey@0"
I20260928 14:46:48.556491 27948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:46:48.568243 27948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:46:48.568615 27948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:46:48.568784 27948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:46:48.573174 27948 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:37565
I20260928 14:46:48.573215 28014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:37565 every 8 connection(s)
I20260928 14:46:48.573608 27948 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:46:48.574599 28015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:46:48.575459 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 27948
I20260928 14:46:48.575618 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:46:48.575891 9349 external_mini_cluster.cc:1428] Setting key 6b244a8ce740648e181040e821b40b30
I20260928 14:46:48.577917 28015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27ec6806cbc84f44a4cda437105f45e5: Bootstrap starting.
I20260928 14:46:48.578568 28015 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27ec6806cbc84f44a4cda437105f45e5: Neither blocks nor log segments found. Creating new log.
I20260928 14:46:48.578883 28015 log.cc:824] T 00000000000000000000000000000000 P 27ec6806cbc84f44a4cda437105f45e5: Log is configured to *not* fsync() on all Append() calls
I20260928 14:46:48.579674 28015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27ec6806cbc84f44a4cda437105f45e5: No bootstrap required, opened a new log
I20260928 14:46:48.581640 28018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:48.576470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38254 (local address 127.9.33.126:37565)
0928 14:46:48.576666 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:48.576674 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:48.576700 (+ 26us) server_negotiation.cc:407] Connection header received
0928 14:46:48.577121 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:48.577128 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:48.577188 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:48.577301 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:48.577547 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:48.578478 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:48.579543 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:48.580071 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:48.580170 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:48.580934 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:48.580938 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:48.580938 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:46:48.581111 (+ 173us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:48.581144 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:46:48.581156 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:48.581206 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:48.581305 (+ 99us) server_negotiation.cc:299] Negotiation successful
0928 14:46:48.581373 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":89,"threads_started":1}
I20260928 14:46:48.582125 28015 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27ec6806cbc84f44a4cda437105f45e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27ec6806cbc84f44a4cda437105f45e5" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37565 } }
I20260928 14:46:48.582262 28015 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27ec6806cbc84f44a4cda437105f45e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:46:48.582304 28015 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27ec6806cbc84f44a4cda437105f45e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27ec6806cbc84f44a4cda437105f45e5, State: Initialized, Role: FOLLOWER
I20260928 14:46:48.582456 28015 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27ec6806cbc84f44a4cda437105f45e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27ec6806cbc84f44a4cda437105f45e5" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37565 } }
I20260928 14:46:48.582528 28015 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27ec6806cbc84f44a4cda437105f45e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:46:48.582551 28015 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27ec6806cbc84f44a4cda437105f45e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:46:48.582582 28015 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27ec6806cbc84f44a4cda437105f45e5 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:46:48.583153 28015 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27ec6806cbc84f44a4cda437105f45e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27ec6806cbc84f44a4cda437105f45e5" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37565 } }
I20260928 14:46:48.583240 28015 leader_election.cc:302] T 00000000000000000000000000000000 P 27ec6806cbc84f44a4cda437105f45e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27ec6806cbc84f44a4cda437105f45e5; no voters:
I20260928 14:46:48.583376 28015 leader_election.cc:288] T 00000000000000000000000000000000 P 27ec6806cbc84f44a4cda437105f45e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:46:48.583485 28020 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27ec6806cbc84f44a4cda437105f45e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:46:48.583690 28020 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27ec6806cbc84f44a4cda437105f45e5 [term 1 LEADER]: Becoming Leader. State: Replica: 27ec6806cbc84f44a4cda437105f45e5, State: Running, Role: LEADER
I20260928 14:46:48.583695 28015 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27ec6806cbc84f44a4cda437105f45e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:46:48.583830 28020 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27ec6806cbc84f44a4cda437105f45e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27ec6806cbc84f44a4cda437105f45e5" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37565 } }
I20260928 14:46:48.584230 28022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27ec6806cbc84f44a4cda437105f45e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27ec6806cbc84f44a4cda437105f45e5. Latest consensus state: current_term: 1 leader_uuid: "27ec6806cbc84f44a4cda437105f45e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27ec6806cbc84f44a4cda437105f45e5" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37565 } } }
I20260928 14:46:48.584327 28022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27ec6806cbc84f44a4cda437105f45e5 [sys.catalog]: This master's current role is: LEADER
I20260928 14:46:48.584492 28021 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27ec6806cbc84f44a4cda437105f45e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27ec6806cbc84f44a4cda437105f45e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27ec6806cbc84f44a4cda437105f45e5" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 37565 } } }
I20260928 14:46:48.584594 28021 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27ec6806cbc84f44a4cda437105f45e5 [sys.catalog]: This master's current role is: LEADER
W20260928 14:46:48.585573 28036 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 27ec6806cbc84f44a4cda437105f45e5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:46:48.585636 28036 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:46:48.585842 28029 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:46:48.586050 28029 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:46:48.587705 28029 catalog_manager.cc:1347] Generated new cluster ID: 36dbe67c8d4c44f4916ba5823806823a
I20260928 14:46:48.587807 28029 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:46:48.608055 28029 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:46:48.608666 28029 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:46:48.615204 28029 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27ec6806cbc84f44a4cda437105f45e5: Generated new TSK 0
I20260928 14:46:48.615463 28029 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:46:48.626200 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.65:0
--local_ip_for_outbound_sockets=127.9.33.65
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:37565
--builtin_ntp_servers=127.9.33.84:36863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 14:46:48.727828 28039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:46:48.728173 28039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:46:48.728214 28039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:46:48.729843 28039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:46:48.729902 28039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:46:48.729931 28039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.65
I20260928 14:46:48.731442 28039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:36863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:37565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.65
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:46:48.731753 28039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:46:48.732101 28039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:46:48.737813 28044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:48.737792 28045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:48.737799 28047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:48.738109 28039 server_base.cc:1034] running on GCE node
I20260928 14:46:48.738370 28039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:46:48.738637 28039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:46:48.739792 28039 hybrid_clock.cc:648] HybridClock initialized: now 1790606808739783 us; error 37 us; skew 500 ppm
I20260928 14:46:48.741137 28039 webserver.cc:460] Webserver started at http://127.9.33.65:40893/ using document root <none> and password file <none>
I20260928 14:46:48.741420 28039 fs_manager.cc:359] Metadata directory not provided
I20260928 14:46:48.741492 28039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:46:48.741642 28039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:46:48.742750 28039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-0/data/instance:
uuid: "10ea0bc1e3064b3596dea3f2e4ae7e8c"
format_stamp: "Formatted at 2026-09-28 14:46:48 on dist-test-slave-pv0w"
server_key: "bc0239066cf6deff78f3ece7654bd448"
server_key_iv: "f366dfa45a0bd7a1dce335b24d62907d"
server_key_version: "encryptionkey@0"
I20260928 14:46:48.743132 28039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance:
uuid: "10ea0bc1e3064b3596dea3f2e4ae7e8c"
format_stamp: "Formatted at 2026-09-28 14:46:48 on dist-test-slave-pv0w"
server_key: "bc0239066cf6deff78f3ece7654bd448"
server_key_iv: "f366dfa45a0bd7a1dce335b24d62907d"
server_key_version: "encryptionkey@0"
I20260928 14:46:48.744582 28039 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:46:48.745366 28053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:48.745539 28039 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:46:48.745645 28039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-0/wal
uuid: "10ea0bc1e3064b3596dea3f2e4ae7e8c"
format_stamp: "Formatted at 2026-09-28 14:46:48 on dist-test-slave-pv0w"
server_key: "bc0239066cf6deff78f3ece7654bd448"
server_key_iv: "f366dfa45a0bd7a1dce335b24d62907d"
server_key_version: "encryptionkey@0"
I20260928 14:46:48.745721 28039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:46:48.763422 28039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:46:48.763837 28039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:46:48.764026 28039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:46:48.764289 28039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:46:48.764649 28039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:46:48.764705 28039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:48.764748 28039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:46:48.764775 28039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:48.770874 28039 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.65:43809
I20260928 14:46:48.770922 28166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.65:43809 every 8 connection(s)
I20260928 14:46:48.771284 28039 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
I20260928 14:46:48.772476 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 28039
I20260928 14:46:48.772598 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance
I20260928 14:46:48.772800 9349 external_mini_cluster.cc:1428] Setting key 9628132c46dcf4d552d9c6cd4f61fe62
I20260928 14:46:48.773895 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.66:0
--local_ip_for_outbound_sockets=127.9.33.66
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:37565
--builtin_ntp_servers=127.9.33.84:36863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 14:46:48.777750 28018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:48.772133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:44159 (local address 127.9.33.126:37565)
0928 14:46:48.772213 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:48.772215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:48.772471 (+ 256us) server_negotiation.cc:407] Connection header received
0928 14:46:48.773424 (+ 953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:48.773424 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:48.773435 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:48.773517 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:48.774453 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:48.774989 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:48.776233 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:48.776435 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:48.776518 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:48.777444 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:48.777446 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:48.777446 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:46:48.777451 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:48.777469 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:46:48.777475 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:48.777542 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:48.777628 (+ 86us) server_negotiation.cc:299] Negotiation successful
0928 14:46:48.777667 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260928 14:46:48.777976 28169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:48.772070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37565 (local address 127.9.33.65:44159)
0928 14:46:48.772306 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:48.772355 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:48.773103 (+ 748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:48.773604 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:48.773608 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:48.773620 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:46:48.774347 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:48.774355 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:48.775127 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:48.775129 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:48.776119 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:48.776122 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:48.776537 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:48.776538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:48.776687 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:48.777277 (+ 590us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:46:48.777277 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:46:48.777310 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:46:48.777334 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:46:48.777544 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:46:48.777551 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:46:48.777554 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:46:48.777631 (+ 77us) client_negotiation.cc:240] Negotiation successful
0928 14:46:48.777747 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":107,"threads_started":1}
I20260928 14:46:48.778594 28167 heartbeater.cc:347] Connected to a master server at 127.9.33.126:37565
I20260928 14:46:48.778704 28167 heartbeater.cc:464] Registering TS with master...
I20260928 14:46:48.778891 28167 heartbeater.cc:511] Master 127.9.33.126:37565 requested a full tablet report, sending...
I20260928 14:46:48.779497 27979 ts_manager.cc:194] Registered new tserver with Master: 10ea0bc1e3064b3596dea3f2e4ae7e8c (127.9.33.65:43809)
I20260928 14:46:48.780568 27979 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.33.65:44159
W20260928 14:46:48.858872 28170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:46:48.859098 28170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:46:48.859122 28170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:46:48.860384 28170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:46:48.860450 28170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:46:48.860478 28170 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.66
I20260928 14:46:48.861851 28170 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:36863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:37565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.66
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:46:48.862068 28170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:46:48.862332 28170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:46:48.867784 28178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:48.867760 28176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:48.867799 28175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:48.868024 28170 server_base.cc:1034] running on GCE node
I20260928 14:46:48.868304 28170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:46:48.868616 28170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:46:48.869783 28170 hybrid_clock.cc:648] HybridClock initialized: now 1790606808869755 us; error 46 us; skew 500 ppm
I20260928 14:46:48.871212 28170 webserver.cc:460] Webserver started at http://127.9.33.66:39233/ using document root <none> and password file <none>
I20260928 14:46:48.871501 28170 fs_manager.cc:359] Metadata directory not provided
I20260928 14:46:48.871598 28170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:46:48.871806 28170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:46:48.872984 28170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-1/data/instance:
uuid: "e1bd4bd634b94082b38bcdbfa274b521"
format_stamp: "Formatted at 2026-09-28 14:46:48 on dist-test-slave-pv0w"
server_key: "62c1c2779cf5ceb6116a2b6ea1b97679"
server_key_iv: "eb3fdc5a5d91815c7bfb6e58898e4ad3"
server_key_version: "encryptionkey@0"
I20260928 14:46:48.873364 28170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance:
uuid: "e1bd4bd634b94082b38bcdbfa274b521"
format_stamp: "Formatted at 2026-09-28 14:46:48 on dist-test-slave-pv0w"
server_key: "62c1c2779cf5ceb6116a2b6ea1b97679"
server_key_iv: "eb3fdc5a5d91815c7bfb6e58898e4ad3"
server_key_version: "encryptionkey@0"
I20260928 14:46:48.874742 28170 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260928 14:46:48.875607 28184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:48.875847 28170 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:46:48.875968 28170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-1/wal
uuid: "e1bd4bd634b94082b38bcdbfa274b521"
format_stamp: "Formatted at 2026-09-28 14:46:48 on dist-test-slave-pv0w"
server_key: "62c1c2779cf5ceb6116a2b6ea1b97679"
server_key_iv: "eb3fdc5a5d91815c7bfb6e58898e4ad3"
server_key_version: "encryptionkey@0"
I20260928 14:46:48.876050 28170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:46:48.885792 28170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:46:48.886152 28170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:46:48.886313 28170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:46:48.886571 28170 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:46:48.886935 28170 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:46:48.886994 28170 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:48.887036 28170 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:46:48.887081 28170 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:48.893034 28170 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.66:45871
I20260928 14:46:48.893074 28297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.66:45871 every 8 connection(s)
I20260928 14:46:48.893466 28170 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
I20260928 14:46:48.899072 28018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:48.894234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:54551 (local address 127.9.33.126:37565)
0928 14:46:48.894303 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:48.894305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:48.894546 (+ 241us) server_negotiation.cc:407] Connection header received
0928 14:46:48.895110 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:48.895112 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:48.895125 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:48.895176 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:48.896022 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:48.896553 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:48.897572 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:48.897793 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:48.897873 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:48.898776 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:48.898778 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:48.898778 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:46:48.898783 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:48.898800 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:46:48.898804 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:48.898856 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:48.898963 (+ 107us) server_negotiation.cc:299] Negotiation successful
0928 14:46:48.898991 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260928 14:46:48.899397 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 28170
I20260928 14:46:48.899488 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance
I20260928 14:46:48.899694 9349 external_mini_cluster.cc:1428] Setting key 48ebe85db6dfe49c3b4001448b935c53
I20260928 14:46:48.899958 28300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:48.894218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37565 (local address 127.9.33.66:54551)
0928 14:46:48.894424 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:48.894458 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:48.894986 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:48.895253 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:48.895257 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:48.895265 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:46:48.895916 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:48.895922 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:48.896672 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:48.896673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:48.897467 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:48.897470 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:48.897881 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:48.897883 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:48.898017 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:48.898615 (+ 598us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:46:48.898615 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:46:48.898645 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:46:48.898668 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:46:48.898867 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:46:48.898871 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:46:48.898875 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:46:48.898953 (+ 78us) client_negotiation.cc:240] Negotiation successful
0928 14:46:48.899060 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":95,"threads_started":1}
I20260928 14:46:48.900388 28298 heartbeater.cc:347] Connected to a master server at 127.9.33.126:37565
I20260928 14:46:48.900544 28298 heartbeater.cc:464] Registering TS with master...
I20260928 14:46:48.900734 28298 heartbeater.cc:511] Master 127.9.33.126:37565 requested a full tablet report, sending...
I20260928 14:46:48.900822 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.67:0
--local_ip_for_outbound_sockets=127.9.33.67
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:37565
--builtin_ntp_servers=127.9.33.84:36863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 14:46:48.901144 27979 ts_manager.cc:194] Registered new tserver with Master: e1bd4bd634b94082b38bcdbfa274b521 (127.9.33.66:45871)
I20260928 14:46:48.901649 27979 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.33.66:54551
W20260928 14:46:48.985345 28301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:46:48.985626 28301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:46:48.985653 28301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:46:48.986860 28301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:46:48.986901 28301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:46:48.986927 28301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.67
I20260928 14:46:48.988422 28301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:36863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:37565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.67
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:46:48.988687 28301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:46:48.989029 28301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 14:46:48.994298 28301 server_base.cc:1034] running on GCE node
W20260928 14:46:48.994272 28307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:48.994360 28306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:48.994271 28309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:48.994769 28301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:46:48.994990 28301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:46:48.996157 28301 hybrid_clock.cc:648] HybridClock initialized: now 1790606808996145 us; error 44 us; skew 500 ppm
I20260928 14:46:48.997503 28301 webserver.cc:460] Webserver started at http://127.9.33.67:46817/ using document root <none> and password file <none>
I20260928 14:46:48.997795 28301 fs_manager.cc:359] Metadata directory not provided
I20260928 14:46:48.997872 28301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:46:48.998049 28301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:46:48.999202 28301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-2/data/instance:
uuid: "a1d8882ce5294890a929d6bbe32430c3"
format_stamp: "Formatted at 2026-09-28 14:46:48 on dist-test-slave-pv0w"
server_key: "f32f92ab274c6184cd156dd66f8e62af"
server_key_iv: "32e5e4fd7245d412aa7457b30c426fe8"
server_key_version: "encryptionkey@0"
I20260928 14:46:48.999565 28301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance:
uuid: "a1d8882ce5294890a929d6bbe32430c3"
format_stamp: "Formatted at 2026-09-28 14:46:48 on dist-test-slave-pv0w"
server_key: "f32f92ab274c6184cd156dd66f8e62af"
server_key_iv: "32e5e4fd7245d412aa7457b30c426fe8"
server_key_version: "encryptionkey@0"
I20260928 14:46:49.001067 28301 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:46:49.001952 28315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:49.002200 28301 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:46:49.002322 28301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-2/wal
uuid: "a1d8882ce5294890a929d6bbe32430c3"
format_stamp: "Formatted at 2026-09-28 14:46:48 on dist-test-slave-pv0w"
server_key: "f32f92ab274c6184cd156dd66f8e62af"
server_key_iv: "32e5e4fd7245d412aa7457b30c426fe8"
server_key_version: "encryptionkey@0"
I20260928 14:46:49.002403 28301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:46:49.020726 28301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:46:49.021104 28301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:46:49.021270 28301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:46:49.021530 28301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:46:49.021931 28301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:46:49.021987 28301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:49.022040 28301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:46:49.022068 28301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:49.028193 28301 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.67:38827
I20260928 14:46:49.028239 28428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.67:38827 every 8 connection(s)
I20260928 14:46:49.028599 28301 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
I20260928 14:46:49.034349 28018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:49.029451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:39955 (local address 127.9.33.126:37565)
0928 14:46:49.029596 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:49.029598 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:49.029780 (+ 182us) server_negotiation.cc:407] Connection header received
0928 14:46:49.030295 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:49.030296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:49.030308 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:49.030375 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:49.031138 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:49.031685 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:49.032885 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:49.033078 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:49.033137 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:49.034050 (+ 913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:49.034052 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:49.034052 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:46:49.034057 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:49.034072 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:46:49.034076 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:49.034127 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:49.034260 (+ 133us) server_negotiation.cc:299] Negotiation successful
0928 14:46:49.034284 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260928 14:46:49.034639 28431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:49.029402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:37565 (local address 127.9.33.67:39955)
0928 14:46:49.029606 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:49.029650 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:49.030157 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:49.030467 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:49.030470 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:49.030480 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:46:49.031034 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:49.031038 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:49.031889 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:49.031890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:49.032784 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:49.032787 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:49.033170 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:49.033171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:49.033293 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:49.033900 (+ 607us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:46:49.033900 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:46:49.033931 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:46:49.033959 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:46:49.034163 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:46:49.034170 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:46:49.034173 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:46:49.034251 (+ 78us) client_negotiation.cc:240] Negotiation successful
0928 14:46:49.034342 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":88,"threads_started":1}
I20260928 14:46:49.035220 28429 heartbeater.cc:347] Connected to a master server at 127.9.33.126:37565
I20260928 14:46:49.035303 28429 heartbeater.cc:464] Registering TS with master...
I20260928 14:46:49.035547 28429 heartbeater.cc:511] Master 127.9.33.126:37565 requested a full tablet report, sending...
I20260928 14:46:49.036065 27979 ts_manager.cc:194] Registered new tserver with Master: a1d8882ce5294890a929d6bbe32430c3 (127.9.33.67:38827)
I20260928 14:46:49.036473 27979 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.33.67:39955
I20260928 14:46:49.036429 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 28301
I20260928 14:46:49.036609 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance
I20260928 14:46:49.036813 9349 external_mini_cluster.cc:1428] Setting key d905b8810d664baee73f47fc45a44885
I20260928 14:46:49.037671 9349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:46:49.042304 28018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:49.039045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38266 (local address 127.9.33.126:37565)
0928 14:46:49.039104 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:49.039122 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:49.039244 (+ 122us) server_negotiation.cc:407] Connection header received
0928 14:46:49.039347 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:49.039348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:49.039358 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:49.039408 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:49.039591 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:49.040156 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:49.040936 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:49.041149 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:49.041227 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:49.041998 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:49.042000 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:49.042000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:46:49.042005 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:49.042019 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:46:49.042023 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:49.042137 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:49.042205 (+ 68us) server_negotiation.cc:299] Negotiation successful
0928 14:46:49.042215 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260928 14:46:49.044111 27979 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38266:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:46:49.045854 27979 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:46:49.052982 28437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:49.048358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:45871 (local address 127.0.0.1:42796)
0928 14:46:49.048518 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:49.048570 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:49.048681 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:49.049128 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:49.049132 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:49.049134 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:46:49.049207 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:49.049209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:49.050546 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:49.050548 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:49.051935 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:49.051938 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:49.052680 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:49.052681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:49.052759 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:49.052771 (+ 12us) client_negotiation.cc:769] Sending connection context
0928 14:46:49.052849 (+ 78us) client_negotiation.cc:240] Negotiation successful
0928 14:46:49.052904 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":79,"thread_start_us":108,"threads_started":1}
I20260928 14:46:49.052995 28438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:49.048499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42796 (local address 127.9.33.66:45871)
0928 14:46:49.048921 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:49.048926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:49.048940 (+ 14us) server_negotiation.cc:407] Connection header received
0928 14:46:49.048988 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:49.048990 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:49.049005 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:49.049092 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:46:49.049284 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:49.050427 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:49.052041 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:49.052586 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:49.052681 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:49.052781 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:49.052862 (+ 81us) server_negotiation.cc:299] Negotiation successful
0928 14:46:49.052900 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":140,"threads_started":1}
I20260928 14:46:49.053215 28439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:49.048614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:38827 (local address 127.0.0.1:34854)
0928 14:46:49.048931 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:49.048941 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:49.049015 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:49.049673 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:49.049674 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:49.049677 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:46:49.049733 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:49.049734 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:49.050904 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:49.050905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:49.051924 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:49.051926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:49.052951 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:49.052952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:49.053027 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:49.053031 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:46:49.053082 (+ 51us) client_negotiation.cc:240] Negotiation successful
0928 14:46:49.053122 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":71,"threads_started":1}
I20260928 14:46:49.053318 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:49.048780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:43809 (local address 127.0.0.1:45018)
0928 14:46:49.049208 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:49.049224 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:49.049304 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:49.049542 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:49.049543 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:49.049545 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:46:49.049592 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:49.049594 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:49.050518 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:49.050527 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:49.052065 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:49.052067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:49.053066 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:49.053067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:49.053120 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:49.053123 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:46:49.053178 (+ 55us) client_negotiation.cc:240] Negotiation successful
0928 14:46:49.053212 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"mutex_wait_us":41,"thread_start_us":97,"threads_started":1}
I20260928 14:46:49.053380 28442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:49.048898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45018 (local address 127.9.33.65:43809)
0928 14:46:49.049153 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:49.049157 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:49.049326 (+ 169us) server_negotiation.cc:407] Connection header received
0928 14:46:49.049376 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:49.049378 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:49.049395 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:49.049469 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:46:49.049671 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:49.050429 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:49.052143 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:49.052891 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:49.052956 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:49.053033 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:49.053224 (+ 191us) server_negotiation.cc:299] Negotiation successful
0928 14:46:49.053275 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":88,"threads_started":1}
I20260928 14:46:49.053469 28440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:49.048722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34854 (local address 127.9.33.67:38827)
0928 14:46:49.048916 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:49.048921 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:49.049005 (+ 84us) server_negotiation.cc:407] Connection header received
0928 14:46:49.049463 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:49.049466 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:49.049480 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:49.049605 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:46:49.049793 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:49.050822 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:49.052013 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:49.052750 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:49.052825 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:49.052913 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:49.053321 (+ 408us) server_negotiation.cc:299] Negotiation successful
0928 14:46:49.053372 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":83,"threads_started":1}
I20260928 14:46:49.054209 28363 tablet_service.cc:1467] Processing CreateTablet for tablet 264770a9d70245169d7313e86ae71209 (DEFAULT_TABLE table=test-table [id=4455cd13af314d82bdb2087a08519bf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:46:49.054209 28101 tablet_service.cc:1467] Processing CreateTablet for tablet 264770a9d70245169d7313e86ae71209 (DEFAULT_TABLE table=test-table [id=4455cd13af314d82bdb2087a08519bf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:46:49.054595 28101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 264770a9d70245169d7313e86ae71209. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:46:49.054595 28363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 264770a9d70245169d7313e86ae71209. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:46:49.058264 28443 tablet_bootstrap.cc:492] T 264770a9d70245169d7313e86ae71209 P 10ea0bc1e3064b3596dea3f2e4ae7e8c: Bootstrap starting.
I20260928 14:46:49.059180 28443 tablet_bootstrap.cc:654] T 264770a9d70245169d7313e86ae71209 P 10ea0bc1e3064b3596dea3f2e4ae7e8c: Neither blocks nor log segments found. Creating new log.
I20260928 14:46:49.059324 28232 tablet_service.cc:1467] Processing CreateTablet for tablet 264770a9d70245169d7313e86ae71209 (DEFAULT_TABLE table=test-table [id=4455cd13af314d82bdb2087a08519bf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:46:49.059564 28443 log.cc:824] T 264770a9d70245169d7313e86ae71209 P 10ea0bc1e3064b3596dea3f2e4ae7e8c: Log is configured to *not* fsync() on all Append() calls
I20260928 14:46:49.059842 28232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 264770a9d70245169d7313e86ae71209. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:46:49.060120 28444 tablet_bootstrap.cc:492] T 264770a9d70245169d7313e86ae71209 P a1d8882ce5294890a929d6bbe32430c3: Bootstrap starting.
I20260928 14:46:49.060925 28444 tablet_bootstrap.cc:654] T 264770a9d70245169d7313e86ae71209 P a1d8882ce5294890a929d6bbe32430c3: Neither blocks nor log segments found. Creating new log.
I20260928 14:46:49.061195 28444 log.cc:824] T 264770a9d70245169d7313e86ae71209 P a1d8882ce5294890a929d6bbe32430c3: Log is configured to *not* fsync() on all Append() calls
I20260928 14:46:49.061980 28444 tablet_bootstrap.cc:492] T 264770a9d70245169d7313e86ae71209 P a1d8882ce5294890a929d6bbe32430c3: No bootstrap required, opened a new log
I20260928 14:46:49.062040 28444 ts_tablet_manager.cc:1397] T 264770a9d70245169d7313e86ae71209 P a1d8882ce5294890a929d6bbe32430c3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:46:49.063169 28447 tablet_bootstrap.cc:492] T 264770a9d70245169d7313e86ae71209 P e1bd4bd634b94082b38bcdbfa274b521: Bootstrap starting.
I20260928 14:46:49.063839 28447 tablet_bootstrap.cc:654] T 264770a9d70245169d7313e86ae71209 P e1bd4bd634b94082b38bcdbfa274b521: Neither blocks nor log segments found. Creating new log.
I20260928 14:46:49.064206 28443 tablet_bootstrap.cc:492] T 264770a9d70245169d7313e86ae71209 P 10ea0bc1e3064b3596dea3f2e4ae7e8c: No bootstrap required, opened a new log
I20260928 14:46:49.064203 28447 log.cc:824] T 264770a9d70245169d7313e86ae71209 P e1bd4bd634b94082b38bcdbfa274b521: Log is configured to *not* fsync() on all Append() calls
I20260928 14:46:49.064289 28443 ts_tablet_manager.cc:1397] T 264770a9d70245169d7313e86ae71209 P 10ea0bc1e3064b3596dea3f2e4ae7e8c: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260928 14:46:49.064672 28444 raft_consensus.cc:348] T 264770a9d70245169d7313e86ae71209 P a1d8882ce5294890a929d6bbe32430c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1bd4bd634b94082b38bcdbfa274b521" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45871 } } peers { permanent_uuid: "a1d8882ce5294890a929d6bbe32430c3" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 38827 } } peers { permanent_uuid: "10ea0bc1e3064b3596dea3f2e4ae7e8c" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 43809 } }
I20260928 14:46:49.064877 28444 raft_consensus.cc:374] T 264770a9d70245169d7313e86ae71209 P a1d8882ce5294890a929d6bbe32430c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:46:49.064929 28444 raft_consensus.cc:729] T 264770a9d70245169d7313e86ae71209 P a1d8882ce5294890a929d6bbe32430c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1d8882ce5294890a929d6bbe32430c3, State: Initialized, Role: FOLLOWER
I20260928 14:46:49.065117 28447 tablet_bootstrap.cc:492] T 264770a9d70245169d7313e86ae71209 P e1bd4bd634b94082b38bcdbfa274b521: No bootstrap required, opened a new log
I20260928 14:46:49.065096 28444 consensus_queue.cc:260] T 264770a9d70245169d7313e86ae71209 P a1d8882ce5294890a929d6bbe32430c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1bd4bd634b94082b38bcdbfa274b521" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45871 } } peers { permanent_uuid: "a1d8882ce5294890a929d6bbe32430c3" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 38827 } } peers { permanent_uuid: "10ea0bc1e3064b3596dea3f2e4ae7e8c" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 43809 } }
I20260928 14:46:49.065199 28447 ts_tablet_manager.cc:1397] T 264770a9d70245169d7313e86ae71209 P e1bd4bd634b94082b38bcdbfa274b521: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260928 14:46:49.065387 28444 ts_tablet_manager.cc:1428] T 264770a9d70245169d7313e86ae71209 P a1d8882ce5294890a929d6bbe32430c3: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260928 14:46:49.065536 28429 heartbeater.cc:503] Master 127.9.33.126:37565 was elected leader, sending a full tablet report...
I20260928 14:46:49.067180 28443 raft_consensus.cc:348] T 264770a9d70245169d7313e86ae71209 P 10ea0bc1e3064b3596dea3f2e4ae7e8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1bd4bd634b94082b38bcdbfa274b521" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45871 } } peers { permanent_uuid: "a1d8882ce5294890a929d6bbe32430c3" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 38827 } } peers { permanent_uuid: "10ea0bc1e3064b3596dea3f2e4ae7e8c" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 43809 } }
I20260928 14:46:49.067373 28443 raft_consensus.cc:374] T 264770a9d70245169d7313e86ae71209 P 10ea0bc1e3064b3596dea3f2e4ae7e8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:46:49.067410 28443 raft_consensus.cc:729] T 264770a9d70245169d7313e86ae71209 P 10ea0bc1e3064b3596dea3f2e4ae7e8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10ea0bc1e3064b3596dea3f2e4ae7e8c, State: Initialized, Role: FOLLOWER
I20260928 14:46:49.067548 28443 consensus_queue.cc:260] T 264770a9d70245169d7313e86ae71209 P 10ea0bc1e3064b3596dea3f2e4ae7e8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1bd4bd634b94082b38bcdbfa274b521" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45871 } } peers { permanent_uuid: "a1d8882ce5294890a929d6bbe32430c3" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 38827 } } peers { permanent_uuid: "10ea0bc1e3064b3596dea3f2e4ae7e8c" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 43809 } }
I20260928 14:46:49.067865 28443 ts_tablet_manager.cc:1428] T 264770a9d70245169d7313e86ae71209 P 10ea0bc1e3064b3596dea3f2e4ae7e8c: Time spent starting tablet: real 0.004s user 0.002s sys 0.002s
I20260928 14:46:49.067915 28167 heartbeater.cc:503] Master 127.9.33.126:37565 was elected leader, sending a full tablet report...
I20260928 14:46:49.067866 28447 raft_consensus.cc:348] T 264770a9d70245169d7313e86ae71209 P e1bd4bd634b94082b38bcdbfa274b521 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1bd4bd634b94082b38bcdbfa274b521" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45871 } } peers { permanent_uuid: "a1d8882ce5294890a929d6bbe32430c3" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 38827 } } peers { permanent_uuid: "10ea0bc1e3064b3596dea3f2e4ae7e8c" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 43809 } }
I20260928 14:46:49.068025 28447 raft_consensus.cc:374] T 264770a9d70245169d7313e86ae71209 P e1bd4bd634b94082b38bcdbfa274b521 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:46:49.068087 28447 raft_consensus.cc:729] T 264770a9d70245169d7313e86ae71209 P e1bd4bd634b94082b38bcdbfa274b521 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1bd4bd634b94082b38bcdbfa274b521, State: Initialized, Role: FOLLOWER
I20260928 14:46:49.068210 28447 consensus_queue.cc:260] T 264770a9d70245169d7313e86ae71209 P e1bd4bd634b94082b38bcdbfa274b521 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1bd4bd634b94082b38bcdbfa274b521" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45871 } } peers { permanent_uuid: "a1d8882ce5294890a929d6bbe32430c3" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 38827 } } peers { permanent_uuid: "10ea0bc1e3064b3596dea3f2e4ae7e8c" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 43809 } }
I20260928 14:46:49.068476 28447 ts_tablet_manager.cc:1428] T 264770a9d70245169d7313e86ae71209 P e1bd4bd634b94082b38bcdbfa274b521: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260928 14:46:49.068711 28298 heartbeater.cc:503] Master 127.9.33.126:37565 was elected leader, sending a full tablet report...
I20260928 14:46:49.142642 28449 raft_consensus.cc:482] T 264770a9d70245169d7313e86ae71209 P a1d8882ce5294890a929d6bbe32430c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:46:49.142768 28449 raft_consensus.cc:504] T 264770a9d70245169d7313e86ae71209 P a1d8882ce5294890a929d6bbe32430c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1bd4bd634b94082b38bcdbfa274b521" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45871 } } peers { permanent_uuid: "a1d8882ce5294890a929d6bbe32430c3" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 38827 } } peers { permanent_uuid: "10ea0bc1e3064b3596dea3f2e4ae7e8c" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 43809 } }
I20260928 14:46:49.143070 28449 leader_election.cc:288] T 264770a9d70245169d7313e86ae71209 P a1d8882ce5294890a929d6bbe32430c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1bd4bd634b94082b38bcdbfa274b521 (127.9.33.66:45871), 10ea0bc1e3064b3596dea3f2e4ae7e8c (127.9.33.65:43809)
W20260928 14:46:49.144052 28299 tablet.cc:2369] T 264770a9d70245169d7313e86ae71209 P e1bd4bd634b94082b38bcdbfa274b521: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:46:49.146384 28438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:49.143508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:46505 (local address 127.9.33.66:45871)
0928 14:46:49.143615 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:49.143618 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:49.143629 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:46:49.143664 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:49.143665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:49.143676 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:49.143782 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:46:49.143990 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:49.144586 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:49.145627 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:49.146036 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:49.146089 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:49.146098 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:49.146267 (+ 169us) server_negotiation.cc:299] Negotiation successful
0928 14:46:49.146310 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260928 14:46:49.146390 28431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:49.143277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:45871 (local address 127.9.33.67:46505)
0928 14:46:49.143332 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:49.143344 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:49.143390 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:49.143795 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:49.143796 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:49.143799 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:46:49.143887 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:49.143888 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:49.144675 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:49.144676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:49.145534 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:49.145537 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:49.146120 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:49.146121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:49.146196 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:49.146199 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:46:49.146258 (+ 59us) client_negotiation.cc:240] Negotiation successful
0928 14:46:49.146294 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":28}
I20260928 14:46:49.146476 28442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:49.143493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:46949 (local address 127.9.33.65:43809)
0928 14:46:49.143648 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:49.143650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:49.143780 (+ 130us) server_negotiation.cc:407] Connection header received
0928 14:46:49.143884 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:49.143885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:49.143899 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:49.143953 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:46:49.144142 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:49.144670 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:49.145711 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:49.146094 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:49.146165 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:49.146175 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:49.146361 (+ 186us) server_negotiation.cc:299] Negotiation successful
0928 14:46:49.146406 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260928 14:46:49.146560 28452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:49.143403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:43809 (local address 127.9.33.67:46949)
0928 14:46:49.143704 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:49.143714 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:49.143807 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:49.143984 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:49.143985 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:49.143986 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:46:49.144069 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:49.144070 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:49.144762 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:49.144763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:49.145620 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:49.145623 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:49.146199 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:49.146199 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:49.146279 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:49.146282 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:46:49.146337 (+ 55us) client_negotiation.cc:240] Negotiation successful
0928 14:46:49.146373 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":3,"thread_start_us":175,"threads_started":1}
I20260928 14:46:49.146806 28252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "264770a9d70245169d7313e86ae71209" candidate_uuid: "a1d8882ce5294890a929d6bbe32430c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1bd4bd634b94082b38bcdbfa274b521" is_pre_election: true
I20260928 14:46:49.146863 28121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "264770a9d70245169d7313e86ae71209" candidate_uuid: "a1d8882ce5294890a929d6bbe32430c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10ea0bc1e3064b3596dea3f2e4ae7e8c" is_pre_election: true
I20260928 14:46:49.146994 28252 raft_consensus.cc:2454] T 264770a9d70245169d7313e86ae71209 P e1bd4bd634b94082b38bcdbfa274b521 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1d8882ce5294890a929d6bbe32430c3 in term 0.
I20260928 14:46:49.147022 28121 raft_consensus.cc:2454] T 264770a9d70245169d7313e86ae71209 P 10ea0bc1e3064b3596dea3f2e4ae7e8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1d8882ce5294890a929d6bbe32430c3 in term 0.
I20260928 14:46:49.147243 28316 leader_election.cc:302] T 264770a9d70245169d7313e86ae71209 P a1d8882ce5294890a929d6bbe32430c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10ea0bc1e3064b3596dea3f2e4ae7e8c, a1d8882ce5294890a929d6bbe32430c3; no voters:
I20260928 14:46:49.147370 28449 raft_consensus.cc:2781] T 264770a9d70245169d7313e86ae71209 P a1d8882ce5294890a929d6bbe32430c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:46:49.147426 28449 raft_consensus.cc:482] T 264770a9d70245169d7313e86ae71209 P a1d8882ce5294890a929d6bbe32430c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:46:49.147457 28449 raft_consensus.cc:3037] T 264770a9d70245169d7313e86ae71209 P a1d8882ce5294890a929d6bbe32430c3 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:46:49.148234 28449 raft_consensus.cc:504] T 264770a9d70245169d7313e86ae71209 P a1d8882ce5294890a929d6bbe32430c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1bd4bd634b94082b38bcdbfa274b521" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45871 } } peers { permanent_uuid: "a1d8882ce5294890a929d6bbe32430c3" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 38827 } } peers { permanent_uuid: "10ea0bc1e3064b3596dea3f2e4ae7e8c" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 43809 } }
I20260928 14:46:49.148375 28449 leader_election.cc:288] T 264770a9d70245169d7313e86ae71209 P a1d8882ce5294890a929d6bbe32430c3 [CANDIDATE]: Term 1 election: Requested vote from peers e1bd4bd634b94082b38bcdbfa274b521 (127.9.33.66:45871), 10ea0bc1e3064b3596dea3f2e4ae7e8c (127.9.33.65:43809)
I20260928 14:46:49.148584 28252 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "264770a9d70245169d7313e86ae71209" candidate_uuid: "a1d8882ce5294890a929d6bbe32430c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1bd4bd634b94082b38bcdbfa274b521"
I20260928 14:46:49.148607 28121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "264770a9d70245169d7313e86ae71209" candidate_uuid: "a1d8882ce5294890a929d6bbe32430c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10ea0bc1e3064b3596dea3f2e4ae7e8c"
I20260928 14:46:49.148679 28252 raft_consensus.cc:3037] T 264770a9d70245169d7313e86ae71209 P e1bd4bd634b94082b38bcdbfa274b521 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:46:49.148705 28121 raft_consensus.cc:3037] T 264770a9d70245169d7313e86ae71209 P 10ea0bc1e3064b3596dea3f2e4ae7e8c [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:46:49.149676 28252 raft_consensus.cc:2454] T 264770a9d70245169d7313e86ae71209 P e1bd4bd634b94082b38bcdbfa274b521 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1d8882ce5294890a929d6bbe32430c3 in term 1.
I20260928 14:46:49.149676 28121 raft_consensus.cc:2454] T 264770a9d70245169d7313e86ae71209 P 10ea0bc1e3064b3596dea3f2e4ae7e8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1d8882ce5294890a929d6bbe32430c3 in term 1.
I20260928 14:46:49.149845 28316 leader_election.cc:302] T 264770a9d70245169d7313e86ae71209 P a1d8882ce5294890a929d6bbe32430c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10ea0bc1e3064b3596dea3f2e4ae7e8c, a1d8882ce5294890a929d6bbe32430c3; no voters:
I20260928 14:46:49.149953 28449 raft_consensus.cc:2781] T 264770a9d70245169d7313e86ae71209 P a1d8882ce5294890a929d6bbe32430c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:46:49.150100 28449 raft_consensus.cc:686] T 264770a9d70245169d7313e86ae71209 P a1d8882ce5294890a929d6bbe32430c3 [term 1 LEADER]: Becoming Leader. State: Replica: a1d8882ce5294890a929d6bbe32430c3, State: Running, Role: LEADER
I20260928 14:46:49.150177 28449 consensus_queue.cc:237] T 264770a9d70245169d7313e86ae71209 P a1d8882ce5294890a929d6bbe32430c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1bd4bd634b94082b38bcdbfa274b521" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45871 } } peers { permanent_uuid: "a1d8882ce5294890a929d6bbe32430c3" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 38827 } } peers { permanent_uuid: "10ea0bc1e3064b3596dea3f2e4ae7e8c" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 43809 } }
I20260928 14:46:49.151067 27979 catalog_manager.cc:5495] T 264770a9d70245169d7313e86ae71209 P a1d8882ce5294890a929d6bbe32430c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a1d8882ce5294890a929d6bbe32430c3 (127.9.33.67). New cstate: current_term: 1 leader_uuid: "a1d8882ce5294890a929d6bbe32430c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e1bd4bd634b94082b38bcdbfa274b521" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1d8882ce5294890a929d6bbe32430c3" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 38827 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "10ea0bc1e3064b3596dea3f2e4ae7e8c" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 43809 } health_report { overall_health: UNKNOWN } } }
I20260928 14:46:49.170190 28440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:49.167709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34866 (local address 127.9.33.67:38827)
0928 14:46:49.167797 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:49.167807 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:49.167816 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:46:49.167851 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:49.167851 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:49.167872 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:49.167954 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:46:49.168109 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:49.168604 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:49.169382 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:49.169591 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:49.169670 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:49.169827 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:46:49.169918 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:46:49.170040 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:49.170071 (+ 31us) server_negotiation.cc:299] Negotiation successful
0928 14:46:49.170079 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260928 14:46:49.172274 28121 raft_consensus.cc:1261] T 264770a9d70245169d7313e86ae71209 P 10ea0bc1e3064b3596dea3f2e4ae7e8c [term 1 FOLLOWER]: Refusing update from remote peer a1d8882ce5294890a929d6bbe32430c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:46:49.172323 28252 raft_consensus.cc:1261] T 264770a9d70245169d7313e86ae71209 P e1bd4bd634b94082b38bcdbfa274b521 [term 1 FOLLOWER]: Refusing update from remote peer a1d8882ce5294890a929d6bbe32430c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:46:49.172742 28453 consensus_queue.cc:1035] T 264770a9d70245169d7313e86ae71209 P a1d8882ce5294890a929d6bbe32430c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10ea0bc1e3064b3596dea3f2e4ae7e8c" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 43809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:46:49.172891 28449 consensus_queue.cc:1035] T 264770a9d70245169d7313e86ae71209 P a1d8882ce5294890a929d6bbe32430c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1bd4bd634b94082b38bcdbfa274b521" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 45871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:46:49.176710 28457 mvcc.cc:205] Tried to move back new op lower bound from 7334325490365956096 to 7334325490279809024. Current Snapshot: MvccSnapshot[applied={T|T < 7334325490365956096 or (T in {7334325490365956096})}]
I20260928 14:46:49.177955 27979 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:46:49.178087 27979 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:46:49.178964 27979 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 27ec6806cbc84f44a4cda437105f45e5: Sending DeleteTablet for 3 replicas of tablet 264770a9d70245169d7313e86ae71209
I20260928 14:46:49.179317 28232 tablet_service.cc:1514] Processing DeleteTablet for tablet 264770a9d70245169d7313e86ae71209 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:46:49 UTC) from {username='slave'} at 127.0.0.1:42796
I20260928 14:46:49.179358 28101 tablet_service.cc:1514] Processing DeleteTablet for tablet 264770a9d70245169d7313e86ae71209 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:46:49 UTC) from {username='slave'} at 127.0.0.1:45018
I20260928 14:46:49.179593 28465 tablet_replica.cc:330] T 264770a9d70245169d7313e86ae71209 P e1bd4bd634b94082b38bcdbfa274b521: stopping tablet replica
I20260928 14:46:49.179620 28363 tablet_service.cc:1514] Processing DeleteTablet for tablet 264770a9d70245169d7313e86ae71209 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:46:49 UTC) from {username='slave'} at 127.0.0.1:34854
I20260928 14:46:49.179703 28465 raft_consensus.cc:2229] T 264770a9d70245169d7313e86ae71209 P e1bd4bd634b94082b38bcdbfa274b521 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:46:49.179858 28465 raft_consensus.cc:2258] T 264770a9d70245169d7313e86ae71209 P e1bd4bd634b94082b38bcdbfa274b521 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:46:49.179893 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 28039
I20260928 14:46:49.180001 28467 tablet_replica.cc:330] T 264770a9d70245169d7313e86ae71209 P a1d8882ce5294890a929d6bbe32430c3: stopping tablet replica
I20260928 14:46:49.180883 28465 ts_tablet_manager.cc:1905] T 264770a9d70245169d7313e86ae71209 P e1bd4bd634b94082b38bcdbfa274b521: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:46:49.180967 28467 raft_consensus.cc:2229] T 264770a9d70245169d7313e86ae71209 P a1d8882ce5294890a929d6bbe32430c3 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:46:49.181095 28467 raft_consensus.cc:2258] T 264770a9d70245169d7313e86ae71209 P a1d8882ce5294890a929d6bbe32430c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:46:49.181497 28467 ts_tablet_manager.cc:1905] T 264770a9d70245169d7313e86ae71209 P a1d8882ce5294890a929d6bbe32430c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:46:49.182951 28465 ts_tablet_manager.cc:1918] T 264770a9d70245169d7313e86ae71209 P e1bd4bd634b94082b38bcdbfa274b521: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:46:49.183022 28465 log.cc:1196] T 264770a9d70245169d7313e86ae71209 P e1bd4bd634b94082b38bcdbfa274b521: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-1/wal/wals/264770a9d70245169d7313e86ae71209
I20260928 14:46:49.183046 28467 ts_tablet_manager.cc:1918] T 264770a9d70245169d7313e86ae71209 P a1d8882ce5294890a929d6bbe32430c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:46:49.183115 28467 log.cc:1196] T 264770a9d70245169d7313e86ae71209 P a1d8882ce5294890a929d6bbe32430c3: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790606561545179-9349-0/minicluster-data/ts-2/wal/wals/264770a9d70245169d7313e86ae71209
I20260928 14:46:49.183291 28465 ts_tablet_manager.cc:1939] T 264770a9d70245169d7313e86ae71209 P e1bd4bd634b94082b38bcdbfa274b521: Deleting consensus metadata
I20260928 14:46:49.183343 28467 ts_tablet_manager.cc:1939] T 264770a9d70245169d7313e86ae71209 P a1d8882ce5294890a929d6bbe32430c3: Deleting consensus metadata
I20260928 14:46:49.183754 27963 catalog_manager.cc:4841] TS e1bd4bd634b94082b38bcdbfa274b521 (127.9.33.66:45871): tablet 264770a9d70245169d7313e86ae71209 (table test-table [id=4455cd13af314d82bdb2087a08519bf5]) successfully deleted
I20260928 14:46:49.183847 27963 catalog_manager.cc:4841] TS a1d8882ce5294890a929d6bbe32430c3 (127.9.33.67:38827): tablet 264770a9d70245169d7313e86ae71209 (table test-table [id=4455cd13af314d82bdb2087a08519bf5]) successfully deleted
W20260928 14:46:49.187878 27963 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.33.65:43809 (error 108)
I20260928 14:46:49.188359 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 28170
I20260928 14:46:49.188480 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:49.188332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:43809 (local address 127.0.0.1:45026)
0928 14:46:49.188401 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:49.188439 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.65:43809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260928 14:46:49.188587 27963 catalog_manager.cc:4584] TS 10ea0bc1e3064b3596dea3f2e4ae7e8c (127.9.33.65:43809): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 264770a9d70245169d7313e86ae71209: Network error: Client connection negotiation failed: client connection to 127.9.33.65:43809: connect: Connection refused (error 111)
I20260928 14:46:49.193476 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 28301
I20260928 14:46:49.197906 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 27948
2026-09-28T14:46:49Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (776 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:46:49 dist-test-slave-pv0w krb5kdc[28471](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:46:49 dist-test-slave-pv0w krb5kdc[28471](info): set up 2 sockets
Sep 28 14:46:49 dist-test-slave-pv0w krb5kdc[28471](info): commencing operation
krb5kdc: starting...
W20260928 14:46:50.240320 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:46:50 dist-test-slave-pv0w krb5kdc[28471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606810, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:46:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:46:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.33.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
I20260928 14:46:50.367229 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:46611
--webserver_interface=127.9.33.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:46717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:46611
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:46:50.450994 28484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:46:50.451217 28484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:46:50.451239 28484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:46:50.452517 28484 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:46:50.452564 28484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:46:50.452579 28484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:46:50.452592 28484 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:46:50.452605 28484 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:46:50.454067 28484 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:46717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:46611
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:46611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:46:50.454270 28484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:46:50.454452 28484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:46:50.459534 28492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:50.459527 28490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:50.459734 28484 server_base.cc:1034] running on GCE node
W20260928 14:46:50.459523 28489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:50.460155 28484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:46:50.460465 28484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:46:50.461629 28484 hybrid_clock.cc:648] HybridClock initialized: now 1790606810461618 us; error 33 us; skew 500 ppm
Sep 28 14:46:50 dist-test-slave-pv0w krb5kdc[28471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606810, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:46:50.464421 28484 init.cc:377] Logged in from keytab as kudu/127.9.33.126@KRBTEST.COM (short username kudu)
I20260928 14:46:50.464871 28484 webserver.cc:460] Webserver started at http://127.9.33.126:40353/ using document root <none> and password file <none>
I20260928 14:46:50.465101 28484 fs_manager.cc:359] Metadata directory not provided
I20260928 14:46:50.465168 28484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:46:50.465313 28484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:46:50.466425 28484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "2578373b4cef440e95bac3c7eadea535"
format_stamp: "Formatted at 2026-09-28 14:46:50 on dist-test-slave-pv0w"
server_key: "072c96a4ed7833aea383b198b6c397fc"
server_key_iv: "8c2d5344e02664bf9be5b8fa4902b3b1"
server_key_version: "encryptionkey@0"
I20260928 14:46:50.466804 28484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "2578373b4cef440e95bac3c7eadea535"
format_stamp: "Formatted at 2026-09-28 14:46:50 on dist-test-slave-pv0w"
server_key: "072c96a4ed7833aea383b198b6c397fc"
server_key_iv: "8c2d5344e02664bf9be5b8fa4902b3b1"
server_key_version: "encryptionkey@0"
I20260928 14:46:50.468272 28484 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:46:50.469121 28499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:50.469304 28484 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:46:50.469406 28484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "2578373b4cef440e95bac3c7eadea535"
format_stamp: "Formatted at 2026-09-28 14:46:50 on dist-test-slave-pv0w"
server_key: "072c96a4ed7833aea383b198b6c397fc"
server_key_iv: "8c2d5344e02664bf9be5b8fa4902b3b1"
server_key_version: "encryptionkey@0"
I20260928 14:46:50.469480 28484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:46:50.483299 28484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:46:50.485298 28484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:46:50.485456 28484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:46:50.489369 28484 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:46611
I20260928 14:46:50.489413 28551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:46611 every 8 connection(s)
I20260928 14:46:50.489794 28484 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:46:50.490733 28552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:46:50.493400 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 28484
I20260928 14:46:50.493512 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:46:50.493691 9349 external_mini_cluster.cc:1428] Setting key 2d06bc8ec752198489a99bb29ce9bdd6
I20260928 14:46:50.493721 28552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2578373b4cef440e95bac3c7eadea535: Bootstrap starting.
I20260928 14:46:50.494437 28552 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2578373b4cef440e95bac3c7eadea535: Neither blocks nor log segments found. Creating new log.
I20260928 14:46:50.494740 28552 log.cc:824] T 00000000000000000000000000000000 P 2578373b4cef440e95bac3c7eadea535: Log is configured to *not* fsync() on all Append() calls
I20260928 14:46:50.495422 28552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2578373b4cef440e95bac3c7eadea535: No bootstrap required, opened a new log
I20260928 14:46:50.497300 28552 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2578373b4cef440e95bac3c7eadea535 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2578373b4cef440e95bac3c7eadea535" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 46611 } }
I20260928 14:46:50.497459 28552 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2578373b4cef440e95bac3c7eadea535 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:46:50.497514 28552 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2578373b4cef440e95bac3c7eadea535 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2578373b4cef440e95bac3c7eadea535, State: Initialized, Role: FOLLOWER
I20260928 14:46:50.497705 28552 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2578373b4cef440e95bac3c7eadea535 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2578373b4cef440e95bac3c7eadea535" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 46611 } }
I20260928 14:46:50.497790 28552 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2578373b4cef440e95bac3c7eadea535 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:46:50.497818 28552 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2578373b4cef440e95bac3c7eadea535 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:46:50.497862 28552 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2578373b4cef440e95bac3c7eadea535 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:46:50.498592 28552 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2578373b4cef440e95bac3c7eadea535 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2578373b4cef440e95bac3c7eadea535" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 46611 } }
I20260928 14:46:50.498714 28552 leader_election.cc:302] T 00000000000000000000000000000000 P 2578373b4cef440e95bac3c7eadea535 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2578373b4cef440e95bac3c7eadea535; no voters:
I20260928 14:46:50.498891 28552 leader_election.cc:288] T 00000000000000000000000000000000 P 2578373b4cef440e95bac3c7eadea535 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:46:50.498942 28557 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2578373b4cef440e95bac3c7eadea535 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:46:50.499094 28557 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2578373b4cef440e95bac3c7eadea535 [term 1 LEADER]: Becoming Leader. State: Replica: 2578373b4cef440e95bac3c7eadea535, State: Running, Role: LEADER
I20260928 14:46:50.499166 28552 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2578373b4cef440e95bac3c7eadea535 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:46:50.499292 28557 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2578373b4cef440e95bac3c7eadea535 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2578373b4cef440e95bac3c7eadea535" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 46611 } }
Sep 28 14:46:50 dist-test-slave-pv0w krb5kdc[28471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606810, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:46:50.500641 28559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2578373b4cef440e95bac3c7eadea535 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2578373b4cef440e95bac3c7eadea535. Latest consensus state: current_term: 1 leader_uuid: "2578373b4cef440e95bac3c7eadea535" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2578373b4cef440e95bac3c7eadea535" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 46611 } } }
I20260928 14:46:50.501150 28559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2578373b4cef440e95bac3c7eadea535 [sys.catalog]: This master's current role is: LEADER
W20260928 14:46:50.501219 28572 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2578373b4cef440e95bac3c7eadea535: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:46:50.501261 28572 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:46:50.501374 28574 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:46:50.501590 28574 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:46:50.500761 28558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2578373b4cef440e95bac3c7eadea535 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2578373b4cef440e95bac3c7eadea535" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2578373b4cef440e95bac3c7eadea535" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 46611 } } }
I20260928 14:46:50.501709 28558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2578373b4cef440e95bac3c7eadea535 [sys.catalog]: This master's current role is: LEADER
I20260928 14:46:50.503170 28574 catalog_manager.cc:1347] Generated new cluster ID: f78f3b57492e4d82a2e8bd69ea0d3985
I20260928 14:46:50.503232 28574 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:46:50.507246 28555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:50.494234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58634 (local address 127.9.33.126:46611)
0928 14:46:50.494399 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:50.494407 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:50.494436 (+ 29us) server_negotiation.cc:407] Connection header received
0928 14:46:50.494950 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:50.494958 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:50.495008 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:50.495157 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:50.495945 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:50.496817 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:50.497619 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:50.498255 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:50.498372 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:50.501470 (+ 3098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:50.501474 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:50.501475 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:46:50.501487 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:50.505328 (+ 3841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:50.505739 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:50.505743 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:50.505743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:50.505847 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:50.506052 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:50.506053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:50.506053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:50.506300 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:50.506919 (+ 619us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:50.506986 (+ 67us) server_negotiation.cc:299] Negotiation successful
0928 14:46:50.507095 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":74,"threads_started":1}
I20260928 14:46:50.522363 28574 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:46:50.523187 28574 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:46:50.526206 28574 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2578373b4cef440e95bac3c7eadea535: Generated new TSK 0
I20260928 14:46:50.526394 28574 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.33.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab.
Entry for principal kudu/127.9.33.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab.
I20260928 14:46:50.552935 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.65:0
--local_ip_for_outbound_sockets=127.9.33.65
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:46611
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:46717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:46:50.635031 28578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:46:50.635253 28578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:46:50.635278 28578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:46:50.636551 28578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:46:50.636600 28578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:46:50.636628 28578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.65
I20260928 14:46:50.638224 28578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:46717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.65.keytab
--principal=kudu/127.9.33.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:46611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.65
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:46:50.638484 28578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:46:50.638729 28578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:46:50.643816 28583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:50.643900 28578 server_base.cc:1034] running on GCE node
W20260928 14:46:50.643827 28586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:50.643827 28584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:50.644201 28578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:46:50.644452 28578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:46:50.645610 28578 hybrid_clock.cc:648] HybridClock initialized: now 1790606810645603 us; error 33 us; skew 500 ppm
Sep 28 14:46:50 dist-test-slave-pv0w krb5kdc[28471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606810, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:46:50.648236 28578 init.cc:377] Logged in from keytab as kudu/127.9.33.65@KRBTEST.COM (short username kudu)
I20260928 14:46:50.648625 28578 webserver.cc:460] Webserver started at http://127.9.33.65:42643/ using document root <none> and password file <none>
I20260928 14:46:50.648818 28578 fs_manager.cc:359] Metadata directory not provided
I20260928 14:46:50.648857 28578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:46:50.648944 28578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:46:50.649942 28578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-0/data/instance:
uuid: "09558b0cca6a463889416317eefdd420"
format_stamp: "Formatted at 2026-09-28 14:46:50 on dist-test-slave-pv0w"
server_key: "77b911721f3ad3f41b5e4fb4f95802d5"
server_key_iv: "4f85c7f6c1d98335872c4246bcd713d0"
server_key_version: "encryptionkey@0"
I20260928 14:46:50.650246 28578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance:
uuid: "09558b0cca6a463889416317eefdd420"
format_stamp: "Formatted at 2026-09-28 14:46:50 on dist-test-slave-pv0w"
server_key: "77b911721f3ad3f41b5e4fb4f95802d5"
server_key_iv: "4f85c7f6c1d98335872c4246bcd713d0"
server_key_version: "encryptionkey@0"
I20260928 14:46:50.651523 28578 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:46:50.652282 28593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:50.652446 28578 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:50.652530 28578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-0/wal
uuid: "09558b0cca6a463889416317eefdd420"
format_stamp: "Formatted at 2026-09-28 14:46:50 on dist-test-slave-pv0w"
server_key: "77b911721f3ad3f41b5e4fb4f95802d5"
server_key_iv: "4f85c7f6c1d98335872c4246bcd713d0"
server_key_version: "encryptionkey@0"
I20260928 14:46:50.652608 28578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:46:50.673447 28578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:46:50.675643 28578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:46:50.675841 28578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:46:50.676090 28578 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:46:50.676416 28578 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:46:50.676469 28578 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:50.676513 28578 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:46:50.676541 28578 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:50.682363 28578 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.65:41581
I20260928 14:46:50.682454 28706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.65:41581 every 8 connection(s)
I20260928 14:46:50.682776 28578 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
I20260928 14:46:50.688917 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 28578
I20260928 14:46:50.689040 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance
Sep 28 14:46:50 dist-test-slave-pv0w krb5kdc[28471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606810, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.65@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:46:50.689254 9349 external_mini_cluster.cc:1428] Setting key 5d933b583510f9de3174659ed37228ff
I20260928 14:46:50.692909 28555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:50.683528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:60821 (local address 127.9.33.126:46611)
0928 14:46:50.683615 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:50.683617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:50.683896 (+ 279us) server_negotiation.cc:407] Connection header received
0928 14:46:50.684414 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:50.684415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:50.684426 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:50.684498 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:50.685463 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:50.685992 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:50.686965 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:50.687175 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:50.687241 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:50.689590 (+ 2349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:50.689592 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:50.689592 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:46:50.689601 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:50.691282 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:50.691830 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:50.691831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:50.691832 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:50.691901 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:50.692166 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:50.692167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:50.692167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:50.692325 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:50.692395 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:50.692664 (+ 269us) server_negotiation.cc:299] Negotiation successful
0928 14:46:50.692768 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260928 14:46:50.692916 28709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:50.683518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46611 (local address 127.9.33.65:60821)
0928 14:46:50.683756 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:50.683804 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:50.684294 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:50.684571 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:50.684574 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:50.685174 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:46:50.685368 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:50.685372 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:50.686071 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:50.686072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:50.686863 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:50.686865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:50.687255 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:50.687256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:50.687356 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:50.687917 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:46:50.687925 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:46:50.689505 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:46:50.691368 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:46:50.691375 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:46:50.691377 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:46:50.691692 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:46:50.691976 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:46:50.691976 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:46:50.691977 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:46:50.692084 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:46:50.692388 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:46:50.692393 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:46:50.692516 (+ 123us) client_negotiation.cc:769] Sending connection context
0928 14:46:50.692593 (+ 77us) client_negotiation.cc:240] Negotiation successful
0928 14:46:50.692748 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":114,"threads_started":1}
I20260928 14:46:50.693439 28707 heartbeater.cc:347] Connected to a master server at 127.9.33.126:46611
I20260928 14:46:50.693569 28707 heartbeater.cc:464] Registering TS with master...
I20260928 14:46:50.693756 28707 heartbeater.cc:511] Master 127.9.33.126:46611 requested a full tablet report, sending...
I20260928 14:46:50.694326 28516 ts_manager.cc:194] Registered new tserver with Master: 09558b0cca6a463889416317eefdd420 (127.9.33.65:41581)
I20260928 14:46:50.695343 28516 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.65@KRBTEST.COM'} at 127.9.33.65:60821
WARNING: no policy specified for kudu/127.9.33.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab.
Entry for principal kudu/127.9.33.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab.
I20260928 14:46:50.719512 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.66:0
--local_ip_for_outbound_sockets=127.9.33.66
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:46611
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:46717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:46:50.804190 28712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:46:50.804418 28712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:46:50.804454 28712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:46:50.805653 28712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:46:50.805723 28712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:46:50.805768 28712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.66
I20260928 14:46:50.807263 28712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:46717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.66.keytab
--principal=kudu/127.9.33.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:46611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.66
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:46:50.807545 28712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:46:50.807816 28712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:46:50.812992 28718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:50.813060 28717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:50.813118 28712 server_base.cc:1034] running on GCE node
W20260928 14:46:50.813060 28720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:50.813431 28712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:46:50.813737 28712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:46:50.814909 28712 hybrid_clock.cc:648] HybridClock initialized: now 1790606810814888 us; error 35 us; skew 500 ppm
Sep 28 14:46:50 dist-test-slave-pv0w krb5kdc[28471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606810, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:46:50.818051 28712 init.cc:377] Logged in from keytab as kudu/127.9.33.66@KRBTEST.COM (short username kudu)
I20260928 14:46:50.818617 28712 webserver.cc:460] Webserver started at http://127.9.33.66:40187/ using document root <none> and password file <none>
I20260928 14:46:50.818876 28712 fs_manager.cc:359] Metadata directory not provided
I20260928 14:46:50.818941 28712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:46:50.819089 28712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:46:50.820276 28712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-1/data/instance:
uuid: "af4206cecdfe47499a68aa849dc33d75"
format_stamp: "Formatted at 2026-09-28 14:46:50 on dist-test-slave-pv0w"
server_key: "4efca8a5985cb9c8b4491d20017b2970"
server_key_iv: "8850c25a848ecc382e82df6b903d8b74"
server_key_version: "encryptionkey@0"
I20260928 14:46:50.820638 28712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance:
uuid: "af4206cecdfe47499a68aa849dc33d75"
format_stamp: "Formatted at 2026-09-28 14:46:50 on dist-test-slave-pv0w"
server_key: "4efca8a5985cb9c8b4491d20017b2970"
server_key_iv: "8850c25a848ecc382e82df6b903d8b74"
server_key_version: "encryptionkey@0"
I20260928 14:46:50.822013 28712 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:46:50.822809 28727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:50.823004 28712 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:46:50.823103 28712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-1/wal
uuid: "af4206cecdfe47499a68aa849dc33d75"
format_stamp: "Formatted at 2026-09-28 14:46:50 on dist-test-slave-pv0w"
server_key: "4efca8a5985cb9c8b4491d20017b2970"
server_key_iv: "8850c25a848ecc382e82df6b903d8b74"
server_key_version: "encryptionkey@0"
I20260928 14:46:50.823181 28712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:46:50.864331 28712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:46:50.866688 28712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:46:50.866882 28712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:46:50.867141 28712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:46:50.867496 28712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:46:50.867548 28712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:50.867584 28712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:46:50.867612 28712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:50.873737 28712 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.66:42449
I20260928 14:46:50.873814 28840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.66:42449 every 8 connection(s)
I20260928 14:46:50.874169 28712 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
I20260928 14:46:50.875162 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 28712
I20260928 14:46:50.875293 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance
I20260928 14:46:50.875504 9349 external_mini_cluster.cc:1428] Setting key 64d6828fb27693e29e63370a2b51035a
Sep 28 14:46:50 dist-test-slave-pv0w krb5kdc[28471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606810, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.66@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.33.67@KRBTEST.COM; defaulting to no policy
I20260928 14:46:50.884837 28555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:50.875045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:43833 (local address 127.9.33.126:46611)
0928 14:46:50.875094 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:50.875096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:50.875392 (+ 296us) server_negotiation.cc:407] Connection header received
0928 14:46:50.876100 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:50.876100 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:50.876131 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:50.876184 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:50.877576 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:50.878166 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:50.879180 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:50.879390 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:50.879476 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:50.881833 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:50.881834 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:50.881835 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:46:50.881843 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:50.883310 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:50.883820 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:50.883822 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:50.883822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:50.883882 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:50.884148 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:50.884149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:50.884150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:50.884284 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:50.884359 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:50.884652 (+ 293us) server_negotiation.cc:299] Negotiation successful
0928 14:46:50.884740 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 14:46:50.885066 28843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:50.875041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46611 (local address 127.9.33.66:43833)
0928 14:46:50.875255 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:50.875308 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:50.876002 (+ 694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:50.876277 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:50.876281 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:50.877168 (+ 887us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:46:50.877459 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:50.877465 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:50.878254 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:50.878255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:50.879075 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:50.879077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:50.879501 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:50.879503 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:50.879653 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:50.880264 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:46:50.880276 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:46:50.881726 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:46:50.883429 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:46:50.883438 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:46:50.883440 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:46:50.883693 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:46:50.883958 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:46:50.883959 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:46:50.883960 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:46:50.884075 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:46:50.884350 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:46:50.884358 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:46:50.884512 (+ 154us) client_negotiation.cc:769] Sending connection context
0928 14:46:50.884616 (+ 104us) client_negotiation.cc:240] Negotiation successful
0928 14:46:50.884792 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":107,"threads_started":1}
I20260928 14:46:50.885761 28841 heartbeater.cc:347] Connected to a master server at 127.9.33.126:46611
I20260928 14:46:50.885859 28841 heartbeater.cc:464] Registering TS with master...
I20260928 14:46:50.886106 28841 heartbeater.cc:511] Master 127.9.33.126:46611 requested a full tablet report, sending...
I20260928 14:46:50.886544 28516 ts_manager.cc:194] Registered new tserver with Master: af4206cecdfe47499a68aa849dc33d75 (127.9.33.66:42449)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.67@KRBTEST.COM" created.
I20260928 14:46:50.887108 28516 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.66@KRBTEST.COM'} at 127.9.33.66:43833
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab.
Entry for principal kudu/127.9.33.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab.
I20260928 14:46:50.902928 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.67:0
--local_ip_for_outbound_sockets=127.9.33.67
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:46611
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:46717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:46:50.985850 28846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:46:50.986079 28846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:46:50.986105 28846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:46:50.987259 28846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:46:50.987298 28846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:46:50.987358 28846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.67
I20260928 14:46:50.988902 28846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:46717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.67.keytab
--principal=kudu/127.9.33.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:46611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.67
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:46:50.989110 28846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:46:50.989296 28846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 14:46:50.994413 28846 server_base.cc:1034] running on GCE node
W20260928 14:46:50.994336 28852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:50.994330 28854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:50.994330 28851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:50.994784 28846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:46:50.995038 28846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:46:50.996203 28846 hybrid_clock.cc:648] HybridClock initialized: now 1790606810996196 us; error 34 us; skew 500 ppm
Sep 28 14:46:50 dist-test-slave-pv0w krb5kdc[28471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606810, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:46:50.998785 28846 init.cc:377] Logged in from keytab as kudu/127.9.33.67@KRBTEST.COM (short username kudu)
I20260928 14:46:50.999279 28846 webserver.cc:460] Webserver started at http://127.9.33.67:39443/ using document root <none> and password file <none>
I20260928 14:46:50.999518 28846 fs_manager.cc:359] Metadata directory not provided
I20260928 14:46:50.999583 28846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:46:50.999773 28846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:46:51.000909 28846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-2/data/instance:
uuid: "db3d7694d02c4b1aae45985c889ae3fa"
format_stamp: "Formatted at 2026-09-28 14:46:50 on dist-test-slave-pv0w"
server_key: "ff066a846fc505c96aad1ec6195d5e53"
server_key_iv: "b8c4d64f003b4a4298b8805e7140d540"
server_key_version: "encryptionkey@0"
I20260928 14:46:51.001288 28846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance:
uuid: "db3d7694d02c4b1aae45985c889ae3fa"
format_stamp: "Formatted at 2026-09-28 14:46:50 on dist-test-slave-pv0w"
server_key: "ff066a846fc505c96aad1ec6195d5e53"
server_key_iv: "b8c4d64f003b4a4298b8805e7140d540"
server_key_version: "encryptionkey@0"
I20260928 14:46:51.002668 28846 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:46:51.003443 28861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:51.003616 28846 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260928 14:46:51.003714 28846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-2/wal
uuid: "db3d7694d02c4b1aae45985c889ae3fa"
format_stamp: "Formatted at 2026-09-28 14:46:50 on dist-test-slave-pv0w"
server_key: "ff066a846fc505c96aad1ec6195d5e53"
server_key_iv: "b8c4d64f003b4a4298b8805e7140d540"
server_key_version: "encryptionkey@0"
I20260928 14:46:51.003829 28846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:46:51.017491 28846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:46:51.019589 28846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:46:51.019774 28846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:46:51.020035 28846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:46:51.020565 28846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:46:51.020627 28846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:51.020668 28846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:46:51.020696 28846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:51.026847 28846 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.67:43669
I20260928 14:46:51.026894 28974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.67:43669 every 8 connection(s)
I20260928 14:46:51.027256 28846 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
I20260928 14:46:51.028031 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 28846
I20260928 14:46:51.028144 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance
I20260928 14:46:51.028364 9349 external_mini_cluster.cc:1428] Setting key d52c40ae45ef2fe3408734ec33777479
Sep 28 14:46:51 dist-test-slave-pv0w krb5kdc[28471](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606810, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.67@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:46:51.037268 28555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.028020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:45693 (local address 127.9.33.126:46611)
0928 14:46:51.028078 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:51.028080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:51.028343 (+ 263us) server_negotiation.cc:407] Connection header received
0928 14:46:51.029042 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:51.029043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:51.029056 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:51.029098 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:51.030153 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.030673 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.031701 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.031928 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.032000 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:51.034331 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:51.034333 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:51.034333 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:46:51.034340 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:51.035859 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:51.036295 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:51.036297 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:51.036297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:51.036356 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:51.036596 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:51.036597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:51.036598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:51.036732 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:51.036811 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:51.037080 (+ 269us) server_negotiation.cc:299] Negotiation successful
0928 14:46:51.037164 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260928 14:46:51.037482 28977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.028012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:46611 (local address 127.9.33.67:45693)
0928 14:46:51.028191 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:51.028241 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:51.028939 (+ 698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:51.029188 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:51.029193 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:51.029829 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:46:51.030049 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:51.030053 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.030772 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.030773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:51.031599 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:51.031601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.032011 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.032013 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:51.032151 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:51.032738 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:46:51.032747 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:46:51.034230 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:46:51.035949 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:46:51.035957 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:46:51.035959 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:46:51.036217 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:46:51.036423 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:46:51.036424 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:46:51.036424 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:46:51.036532 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:46:51.036802 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:46:51.036808 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:46:51.036954 (+ 146us) client_negotiation.cc:769] Sending connection context
0928 14:46:51.037062 (+ 108us) client_negotiation.cc:240] Negotiation successful
0928 14:46:51.037240 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":88,"threads_started":1}
I20260928 14:46:51.038000 28975 heartbeater.cc:347] Connected to a master server at 127.9.33.126:46611
I20260928 14:46:51.038092 28975 heartbeater.cc:464] Registering TS with master...
I20260928 14:46:51.038249 28975 heartbeater.cc:511] Master 127.9.33.126:46611 requested a full tablet report, sending...
I20260928 14:46:51.038605 28516 ts_manager.cc:194] Registered new tserver with Master: db3d7694d02c4b1aae45985c889ae3fa (127.9.33.67:43669)
I20260928 14:46:51.038985 28516 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.33.67@KRBTEST.COM'} at 127.9.33.67:45693
I20260928 14:46:51.040306 9349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:46:51.047313 28555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.041694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58650 (local address 127.9.33.126:46611)
0928 14:46:51.041752 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:51.041754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:51.041874 (+ 120us) server_negotiation.cc:407] Connection header received
0928 14:46:51.041913 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:51.041914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:51.041922 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:51.041979 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:51.042520 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.043003 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.043796 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.043999 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.044055 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:51.045201 (+ 1146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:51.045202 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:51.045202 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:46:51.045206 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:51.046376 (+ 1170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:51.046700 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:51.046702 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:51.046702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:51.046749 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:51.046905 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:51.046905 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:51.046906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:51.047036 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:51.047113 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:51.047197 (+ 84us) server_negotiation.cc:299] Negotiation successful
0928 14:46:51.047252 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260928 14:46:51.049008 28516 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58650:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:46:51.050565 28516 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:46:51.057543 28987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.053134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:41581 (local address 127.0.0.1:51408)
0928 14:46:51.053374 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:51.053418 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:51.053485 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:51.053879 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:51.053883 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:51.053886 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:46:51.053970 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:51.053974 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.055408 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.055409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:51.056487 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:51.056490 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.057223 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.057224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:51.057303 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:51.057318 (+ 15us) client_negotiation.cc:769] Sending connection context
0928 14:46:51.057401 (+ 83us) client_negotiation.cc:240] Negotiation successful
0928 14:46:51.057457 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":69,"threads_started":1}
I20260928 14:46:51.057560 28988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.053279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51408 (local address 127.9.33.65:41581)
0928 14:46:51.053699 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:51.053704 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:51.053715 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:46:51.053758 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:51.053760 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:51.053777 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:51.053859 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:46:51.054062 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.055260 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.056574 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.057139 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.057216 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:51.057277 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:51.057423 (+ 146us) server_negotiation.cc:299] Negotiation successful
0928 14:46:51.057464 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":91,"threads_started":1}
I20260928 14:46:51.057741 28989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.053348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:43669 (local address 127.0.0.1:37078)
0928 14:46:51.053965 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:51.053976 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:51.054021 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:51.054174 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:51.054175 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:51.054176 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:46:51.054223 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:51.054225 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.055145 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.055146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:51.056551 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:51.056554 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.057502 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.057503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:51.057557 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:51.057563 (+ 6us) client_negotiation.cc:769] Sending connection context
0928 14:46:51.057624 (+ 61us) client_negotiation.cc:240] Negotiation successful
0928 14:46:51.057667 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":73,"threads_started":1}
I20260928 14:46:51.058046 28991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.053587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37078 (local address 127.9.33.67:43669)
0928 14:46:51.054004 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:51.054008 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:51.054018 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:46:51.054087 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:51.054088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:51.054102 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:51.054170 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:46:51.054276 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.055049 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.056648 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.057356 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.057417 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:51.057470 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:51.057907 (+ 437us) server_negotiation.cc:299] Negotiation successful
0928 14:46:51.057959 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":95,"threads_started":1}
I20260928 14:46:51.058302 28641 tablet_service.cc:1467] Processing CreateTablet for tablet e37a9107235b4278bbc17d22e66fc12a (DEFAULT_TABLE table=test-table [id=b502409877744b29a29635e53a5b2c42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:46:51.058548 28986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.053222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:42449 (local address 127.0.0.1:54454)
0928 14:46:51.053468 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:51.053480 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:51.053535 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:51.053921 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:51.053922 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:51.053923 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:46:51.053970 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:51.053973 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.055366 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.055367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:51.056946 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:51.056949 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.058291 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.058292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:51.058357 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:51.058366 (+ 9us) client_negotiation.cc:769] Sending connection context
0928 14:46:51.058435 (+ 69us) client_negotiation.cc:240] Negotiation successful
0928 14:46:51.058476 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":5,"thread_start_us":96,"threads_started":1}
I20260928 14:46:51.058733 28641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e37a9107235b4278bbc17d22e66fc12a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:46:51.058923 28990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.053473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54454 (local address 127.9.33.66:42449)
0928 14:46:51.053726 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:51.053730 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:51.053741 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:46:51.053782 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:51.053784 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:51.053798 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:51.054060 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:46:51.054137 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.055260 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.057045 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.058201 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.058281 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:51.058336 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:51.058456 (+ 120us) server_negotiation.cc:299] Negotiation successful
0928 14:46:51.058567 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":79,"threads_started":1}
I20260928 14:46:51.058912 28909 tablet_service.cc:1467] Processing CreateTablet for tablet e37a9107235b4278bbc17d22e66fc12a (DEFAULT_TABLE table=test-table [id=b502409877744b29a29635e53a5b2c42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:46:51.059237 28909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e37a9107235b4278bbc17d22e66fc12a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:46:51.059495 28775 tablet_service.cc:1467] Processing CreateTablet for tablet e37a9107235b4278bbc17d22e66fc12a (DEFAULT_TABLE table=test-table [id=b502409877744b29a29635e53a5b2c42]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:46:51.059818 28775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e37a9107235b4278bbc17d22e66fc12a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:46:51.062453 28992 tablet_bootstrap.cc:492] T e37a9107235b4278bbc17d22e66fc12a P 09558b0cca6a463889416317eefdd420: Bootstrap starting.
I20260928 14:46:51.062688 28993 tablet_bootstrap.cc:492] T e37a9107235b4278bbc17d22e66fc12a P af4206cecdfe47499a68aa849dc33d75: Bootstrap starting.
I20260928 14:46:51.062688 28994 tablet_bootstrap.cc:492] T e37a9107235b4278bbc17d22e66fc12a P db3d7694d02c4b1aae45985c889ae3fa: Bootstrap starting.
I20260928 14:46:51.063098 28992 tablet_bootstrap.cc:654] T e37a9107235b4278bbc17d22e66fc12a P 09558b0cca6a463889416317eefdd420: Neither blocks nor log segments found. Creating new log.
I20260928 14:46:51.063345 28994 tablet_bootstrap.cc:654] T e37a9107235b4278bbc17d22e66fc12a P db3d7694d02c4b1aae45985c889ae3fa: Neither blocks nor log segments found. Creating new log.
I20260928 14:46:51.063345 28993 tablet_bootstrap.cc:654] T e37a9107235b4278bbc17d22e66fc12a P af4206cecdfe47499a68aa849dc33d75: Neither blocks nor log segments found. Creating new log.
I20260928 14:46:51.063658 28992 log.cc:824] T e37a9107235b4278bbc17d22e66fc12a P 09558b0cca6a463889416317eefdd420: Log is configured to *not* fsync() on all Append() calls
I20260928 14:46:51.064015 28994 log.cc:824] T e37a9107235b4278bbc17d22e66fc12a P db3d7694d02c4b1aae45985c889ae3fa: Log is configured to *not* fsync() on all Append() calls
I20260928 14:46:51.064015 28993 log.cc:824] T e37a9107235b4278bbc17d22e66fc12a P af4206cecdfe47499a68aa849dc33d75: Log is configured to *not* fsync() on all Append() calls
I20260928 14:46:51.064620 28992 tablet_bootstrap.cc:492] T e37a9107235b4278bbc17d22e66fc12a P 09558b0cca6a463889416317eefdd420: No bootstrap required, opened a new log
I20260928 14:46:51.064702 28992 ts_tablet_manager.cc:1397] T e37a9107235b4278bbc17d22e66fc12a P 09558b0cca6a463889416317eefdd420: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:46:51.064704 28993 tablet_bootstrap.cc:492] T e37a9107235b4278bbc17d22e66fc12a P af4206cecdfe47499a68aa849dc33d75: No bootstrap required, opened a new log
I20260928 14:46:51.064734 28994 tablet_bootstrap.cc:492] T e37a9107235b4278bbc17d22e66fc12a P db3d7694d02c4b1aae45985c889ae3fa: No bootstrap required, opened a new log
I20260928 14:46:51.064769 28993 ts_tablet_manager.cc:1397] T e37a9107235b4278bbc17d22e66fc12a P af4206cecdfe47499a68aa849dc33d75: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:46:51.064796 28994 ts_tablet_manager.cc:1397] T e37a9107235b4278bbc17d22e66fc12a P db3d7694d02c4b1aae45985c889ae3fa: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:46:51.066362 28993 raft_consensus.cc:348] T e37a9107235b4278bbc17d22e66fc12a P af4206cecdfe47499a68aa849dc33d75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09558b0cca6a463889416317eefdd420" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41581 } } peers { permanent_uuid: "db3d7694d02c4b1aae45985c889ae3fa" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 43669 } } peers { permanent_uuid: "af4206cecdfe47499a68aa849dc33d75" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 42449 } }
I20260928 14:46:51.066548 28993 raft_consensus.cc:374] T e37a9107235b4278bbc17d22e66fc12a P af4206cecdfe47499a68aa849dc33d75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:46:51.066596 28993 raft_consensus.cc:729] T e37a9107235b4278bbc17d22e66fc12a P af4206cecdfe47499a68aa849dc33d75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af4206cecdfe47499a68aa849dc33d75, State: Initialized, Role: FOLLOWER
I20260928 14:46:51.066713 28993 consensus_queue.cc:260] T e37a9107235b4278bbc17d22e66fc12a P af4206cecdfe47499a68aa849dc33d75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09558b0cca6a463889416317eefdd420" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41581 } } peers { permanent_uuid: "db3d7694d02c4b1aae45985c889ae3fa" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 43669 } } peers { permanent_uuid: "af4206cecdfe47499a68aa849dc33d75" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 42449 } }
I20260928 14:46:51.066706 28994 raft_consensus.cc:348] T e37a9107235b4278bbc17d22e66fc12a P db3d7694d02c4b1aae45985c889ae3fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09558b0cca6a463889416317eefdd420" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41581 } } peers { permanent_uuid: "db3d7694d02c4b1aae45985c889ae3fa" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 43669 } } peers { permanent_uuid: "af4206cecdfe47499a68aa849dc33d75" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 42449 } }
I20260928 14:46:51.066706 28992 raft_consensus.cc:348] T e37a9107235b4278bbc17d22e66fc12a P 09558b0cca6a463889416317eefdd420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09558b0cca6a463889416317eefdd420" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41581 } } peers { permanent_uuid: "db3d7694d02c4b1aae45985c889ae3fa" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 43669 } } peers { permanent_uuid: "af4206cecdfe47499a68aa849dc33d75" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 42449 } }
I20260928 14:46:51.066877 28992 raft_consensus.cc:374] T e37a9107235b4278bbc17d22e66fc12a P 09558b0cca6a463889416317eefdd420 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:46:51.066877 28994 raft_consensus.cc:374] T e37a9107235b4278bbc17d22e66fc12a P db3d7694d02c4b1aae45985c889ae3fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:46:51.066919 28992 raft_consensus.cc:729] T e37a9107235b4278bbc17d22e66fc12a P 09558b0cca6a463889416317eefdd420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09558b0cca6a463889416317eefdd420, State: Initialized, Role: FOLLOWER
I20260928 14:46:51.066919 28994 raft_consensus.cc:729] T e37a9107235b4278bbc17d22e66fc12a P db3d7694d02c4b1aae45985c889ae3fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db3d7694d02c4b1aae45985c889ae3fa, State: Initialized, Role: FOLLOWER
I20260928 14:46:51.067005 28993 ts_tablet_manager.cc:1428] T e37a9107235b4278bbc17d22e66fc12a P af4206cecdfe47499a68aa849dc33d75: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260928 14:46:51.067032 28994 consensus_queue.cc:260] T e37a9107235b4278bbc17d22e66fc12a P db3d7694d02c4b1aae45985c889ae3fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09558b0cca6a463889416317eefdd420" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41581 } } peers { permanent_uuid: "db3d7694d02c4b1aae45985c889ae3fa" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 43669 } } peers { permanent_uuid: "af4206cecdfe47499a68aa849dc33d75" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 42449 } }
I20260928 14:46:51.067055 28992 consensus_queue.cc:260] T e37a9107235b4278bbc17d22e66fc12a P 09558b0cca6a463889416317eefdd420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09558b0cca6a463889416317eefdd420" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41581 } } peers { permanent_uuid: "db3d7694d02c4b1aae45985c889ae3fa" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 43669 } } peers { permanent_uuid: "af4206cecdfe47499a68aa849dc33d75" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 42449 } }
I20260928 14:46:51.067252 28841 heartbeater.cc:503] Master 127.9.33.126:46611 was elected leader, sending a full tablet report...
I20260928 14:46:51.067328 28994 ts_tablet_manager.cc:1428] T e37a9107235b4278bbc17d22e66fc12a P db3d7694d02c4b1aae45985c889ae3fa: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260928 14:46:51.067328 28992 ts_tablet_manager.cc:1428] T e37a9107235b4278bbc17d22e66fc12a P 09558b0cca6a463889416317eefdd420: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260928 14:46:51.067446 28707 heartbeater.cc:503] Master 127.9.33.126:46611 was elected leader, sending a full tablet report...
I20260928 14:46:51.067497 28975 heartbeater.cc:503] Master 127.9.33.126:46611 was elected leader, sending a full tablet report...
I20260928 14:46:51.119388 29000 raft_consensus.cc:482] T e37a9107235b4278bbc17d22e66fc12a P 09558b0cca6a463889416317eefdd420 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:46:51.119594 29000 raft_consensus.cc:504] T e37a9107235b4278bbc17d22e66fc12a P 09558b0cca6a463889416317eefdd420 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09558b0cca6a463889416317eefdd420" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41581 } } peers { permanent_uuid: "db3d7694d02c4b1aae45985c889ae3fa" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 43669 } } peers { permanent_uuid: "af4206cecdfe47499a68aa849dc33d75" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 42449 } }
I20260928 14:46:51.120069 29000 leader_election.cc:288] T e37a9107235b4278bbc17d22e66fc12a P 09558b0cca6a463889416317eefdd420 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db3d7694d02c4b1aae45985c889ae3fa (127.9.33.67:43669), af4206cecdfe47499a68aa849dc33d75 (127.9.33.66:42449)
I20260928 14:46:51.123096 28990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.120392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:49543 (local address 127.9.33.66:42449)
0928 14:46:51.120464 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:51.120466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:51.120477 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:46:51.120506 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:51.120507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:51.120524 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:51.120583 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:46:51.120776 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.121341 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.122302 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.122710 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.122765 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:51.122774 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:51.122960 (+ 186us) server_negotiation.cc:299] Negotiation successful
0928 14:46:51.123007 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260928 14:46:51.123112 28709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.120254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:42449 (local address 127.9.33.65:49543)
0928 14:46:51.120349 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:51.120363 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:51.120435 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:51.120589 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:51.120590 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:51.120591 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:46:51.120676 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:51.120693 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.121426 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.121427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:51.122210 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:51.122212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.122791 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.122792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:51.122880 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:51.122883 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:46:51.122933 (+ 50us) client_negotiation.cc:240] Negotiation successful
0928 14:46:51.123000 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260928 14:46:51.123108 28991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.120338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:36277 (local address 127.9.33.67:43669)
0928 14:46:51.120416 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:51.120419 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:51.120711 (+ 292us) server_negotiation.cc:407] Connection header received
0928 14:46:51.120742 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:51.120743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:51.120757 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:51.120810 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:46:51.120940 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.121431 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.122360 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.122729 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.122816 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:51.122826 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:51.122997 (+ 171us) server_negotiation.cc:299] Negotiation successful
0928 14:46:51.123035 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260928 14:46:51.123138 29001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.120258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:43669 (local address 127.9.33.65:36277)
0928 14:46:51.120626 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:51.120636 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:51.120698 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:51.120812 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:51.120812 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:51.120814 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:46:51.120878 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:51.120880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.121514 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.121515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:51.122266 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:51.122268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.122836 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.122837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:51.122901 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:51.122904 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:46:51.122984 (+ 80us) client_negotiation.cc:240] Negotiation successful
0928 14:46:51.123035 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":166,"thread_start_us":141,"threads_started":1}
I20260928 14:46:51.123497 28795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e37a9107235b4278bbc17d22e66fc12a" candidate_uuid: "09558b0cca6a463889416317eefdd420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af4206cecdfe47499a68aa849dc33d75" is_pre_election: true
I20260928 14:46:51.123687 28795 raft_consensus.cc:2454] T e37a9107235b4278bbc17d22e66fc12a P af4206cecdfe47499a68aa849dc33d75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09558b0cca6a463889416317eefdd420 in term 0.
I20260928 14:46:51.123746 28929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e37a9107235b4278bbc17d22e66fc12a" candidate_uuid: "09558b0cca6a463889416317eefdd420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db3d7694d02c4b1aae45985c889ae3fa" is_pre_election: true
I20260928 14:46:51.123912 28929 raft_consensus.cc:2454] T e37a9107235b4278bbc17d22e66fc12a P db3d7694d02c4b1aae45985c889ae3fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09558b0cca6a463889416317eefdd420 in term 0.
I20260928 14:46:51.123960 28596 leader_election.cc:302] T e37a9107235b4278bbc17d22e66fc12a P 09558b0cca6a463889416317eefdd420 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09558b0cca6a463889416317eefdd420, af4206cecdfe47499a68aa849dc33d75; no voters:
I20260928 14:46:51.124125 29000 raft_consensus.cc:2781] T e37a9107235b4278bbc17d22e66fc12a P 09558b0cca6a463889416317eefdd420 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:46:51.124176 29000 raft_consensus.cc:482] T e37a9107235b4278bbc17d22e66fc12a P 09558b0cca6a463889416317eefdd420 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:46:51.124209 29000 raft_consensus.cc:3037] T e37a9107235b4278bbc17d22e66fc12a P 09558b0cca6a463889416317eefdd420 [term 0 FOLLOWER]: Advancing to term 1
W20260928 14:46:51.124747 28842 tablet.cc:2369] T e37a9107235b4278bbc17d22e66fc12a P af4206cecdfe47499a68aa849dc33d75: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260928 14:46:51.125124 29000 raft_consensus.cc:504] T e37a9107235b4278bbc17d22e66fc12a P 09558b0cca6a463889416317eefdd420 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09558b0cca6a463889416317eefdd420" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41581 } } peers { permanent_uuid: "db3d7694d02c4b1aae45985c889ae3fa" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 43669 } } peers { permanent_uuid: "af4206cecdfe47499a68aa849dc33d75" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 42449 } }
I20260928 14:46:51.125273 29000 leader_election.cc:288] T e37a9107235b4278bbc17d22e66fc12a P 09558b0cca6a463889416317eefdd420 [CANDIDATE]: Term 1 election: Requested vote from peers db3d7694d02c4b1aae45985c889ae3fa (127.9.33.67:43669), af4206cecdfe47499a68aa849dc33d75 (127.9.33.66:42449)
I20260928 14:46:51.125461 28795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e37a9107235b4278bbc17d22e66fc12a" candidate_uuid: "09558b0cca6a463889416317eefdd420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af4206cecdfe47499a68aa849dc33d75"
I20260928 14:46:51.125519 28929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e37a9107235b4278bbc17d22e66fc12a" candidate_uuid: "09558b0cca6a463889416317eefdd420" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db3d7694d02c4b1aae45985c889ae3fa"
I20260928 14:46:51.125550 28795 raft_consensus.cc:3037] T e37a9107235b4278bbc17d22e66fc12a P af4206cecdfe47499a68aa849dc33d75 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:46:51.125602 28929 raft_consensus.cc:3037] T e37a9107235b4278bbc17d22e66fc12a P db3d7694d02c4b1aae45985c889ae3fa [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:46:51.126436 28929 raft_consensus.cc:2454] T e37a9107235b4278bbc17d22e66fc12a P db3d7694d02c4b1aae45985c889ae3fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09558b0cca6a463889416317eefdd420 in term 1.
I20260928 14:46:51.126466 28795 raft_consensus.cc:2454] T e37a9107235b4278bbc17d22e66fc12a P af4206cecdfe47499a68aa849dc33d75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09558b0cca6a463889416317eefdd420 in term 1.
I20260928 14:46:51.126644 28594 leader_election.cc:302] T e37a9107235b4278bbc17d22e66fc12a P 09558b0cca6a463889416317eefdd420 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09558b0cca6a463889416317eefdd420, db3d7694d02c4b1aae45985c889ae3fa; no voters:
I20260928 14:46:51.126751 29000 raft_consensus.cc:2781] T e37a9107235b4278bbc17d22e66fc12a P 09558b0cca6a463889416317eefdd420 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:46:51.126917 29000 raft_consensus.cc:686] T e37a9107235b4278bbc17d22e66fc12a P 09558b0cca6a463889416317eefdd420 [term 1 LEADER]: Becoming Leader. State: Replica: 09558b0cca6a463889416317eefdd420, State: Running, Role: LEADER
I20260928 14:46:51.127018 29000 consensus_queue.cc:237] T e37a9107235b4278bbc17d22e66fc12a P 09558b0cca6a463889416317eefdd420 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09558b0cca6a463889416317eefdd420" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41581 } } peers { permanent_uuid: "db3d7694d02c4b1aae45985c889ae3fa" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 43669 } } peers { permanent_uuid: "af4206cecdfe47499a68aa849dc33d75" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 42449 } }
I20260928 14:46:51.127923 28516 catalog_manager.cc:5495] T e37a9107235b4278bbc17d22e66fc12a P 09558b0cca6a463889416317eefdd420 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09558b0cca6a463889416317eefdd420 (127.9.33.65). New cstate: current_term: 1 leader_uuid: "09558b0cca6a463889416317eefdd420" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09558b0cca6a463889416317eefdd420" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41581 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db3d7694d02c4b1aae45985c889ae3fa" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 43669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af4206cecdfe47499a68aa849dc33d75" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 42449 } health_report { overall_health: UNKNOWN } } }
I20260928 14:46:51.148841 28988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.146338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51412 (local address 127.9.33.65:41581)
0928 14:46:51.146398 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:51.146400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:51.146407 (+ 7us) server_negotiation.cc:407] Connection header received
0928 14:46:51.146442 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:51.146443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:51.146463 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:51.146595 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:46:51.146747 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.147225 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.148058 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.148253 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.148334 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:51.148512 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:46:51.148616 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:46:51.148701 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:51.148725 (+ 24us) server_negotiation.cc:299] Negotiation successful
0928 14:46:51.148749 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260928 14:46:51.150852 28795 raft_consensus.cc:1261] T e37a9107235b4278bbc17d22e66fc12a P af4206cecdfe47499a68aa849dc33d75 [term 1 FOLLOWER]: Refusing update from remote peer 09558b0cca6a463889416317eefdd420: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:46:51.151015 28929 raft_consensus.cc:1261] T e37a9107235b4278bbc17d22e66fc12a P db3d7694d02c4b1aae45985c889ae3fa [term 1 FOLLOWER]: Refusing update from remote peer 09558b0cca6a463889416317eefdd420: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:46:51.151273 29002 consensus_queue.cc:1035] T e37a9107235b4278bbc17d22e66fc12a P 09558b0cca6a463889416317eefdd420 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af4206cecdfe47499a68aa849dc33d75" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 42449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:46:51.151401 29000 consensus_queue.cc:1035] T e37a9107235b4278bbc17d22e66fc12a P 09558b0cca6a463889416317eefdd420 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db3d7694d02c4b1aae45985c889ae3fa" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 43669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:46:51.154030 29005 mvcc.cc:205] Tried to move back new op lower bound from 7334325498470674432 to 7334325498377035776. Current Snapshot: MvccSnapshot[applied={T|T < 7334325498470674432}]
I20260928 14:46:51.154884 29007 mvcc.cc:205] Tried to move back new op lower bound from 7334325498470674432 to 7334325498377035776. Current Snapshot: MvccSnapshot[applied={T|T < 7334325498470674432}]
I20260928 14:46:51.156584 28515 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:46:51.156701 28515 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:46:51.157428 28515 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2578373b4cef440e95bac3c7eadea535: Sending DeleteTablet for 3 replicas of tablet e37a9107235b4278bbc17d22e66fc12a
I20260928 14:46:51.157787 28641 tablet_service.cc:1514] Processing DeleteTablet for tablet e37a9107235b4278bbc17d22e66fc12a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:46:51 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:51408
I20260928 14:46:51.157823 28909 tablet_service.cc:1514] Processing DeleteTablet for tablet e37a9107235b4278bbc17d22e66fc12a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:46:51 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:37078
I20260928 14:46:51.157896 28775 tablet_service.cc:1514] Processing DeleteTablet for tablet e37a9107235b4278bbc17d22e66fc12a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:46:51 UTC) from {username='kudu', principal='kudu/127.9.33.126@KRBTEST.COM'} at 127.0.0.1:54454
I20260928 14:46:51.158150 29015 tablet_replica.cc:330] T e37a9107235b4278bbc17d22e66fc12a P db3d7694d02c4b1aae45985c889ae3fa: stopping tablet replica
I20260928 14:46:51.158211 29014 tablet_replica.cc:330] T e37a9107235b4278bbc17d22e66fc12a P 09558b0cca6a463889416317eefdd420: stopping tablet replica
I20260928 14:46:51.158318 29014 raft_consensus.cc:2229] T e37a9107235b4278bbc17d22e66fc12a P 09558b0cca6a463889416317eefdd420 [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:46:51.158319 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 28578
I20260928 14:46:51.158420 29014 raft_consensus.cc:2258] T e37a9107235b4278bbc17d22e66fc12a P 09558b0cca6a463889416317eefdd420 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:46:51.158342 29016 tablet_replica.cc:330] T e37a9107235b4278bbc17d22e66fc12a P af4206cecdfe47499a68aa849dc33d75: stopping tablet replica
I20260928 14:46:51.159116 29015 raft_consensus.cc:2229] T e37a9107235b4278bbc17d22e66fc12a P db3d7694d02c4b1aae45985c889ae3fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:46:51.159256 29016 raft_consensus.cc:2229] T e37a9107235b4278bbc17d22e66fc12a P af4206cecdfe47499a68aa849dc33d75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:46:51.159408 29015 raft_consensus.cc:2258] T e37a9107235b4278bbc17d22e66fc12a P db3d7694d02c4b1aae45985c889ae3fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:46:51.159538 29016 raft_consensus.cc:2258] T e37a9107235b4278bbc17d22e66fc12a P af4206cecdfe47499a68aa849dc33d75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:46:51.160207 29016 ts_tablet_manager.cc:1905] T e37a9107235b4278bbc17d22e66fc12a P af4206cecdfe47499a68aa849dc33d75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:46:51.162240 29016 ts_tablet_manager.cc:1918] T e37a9107235b4278bbc17d22e66fc12a P af4206cecdfe47499a68aa849dc33d75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:46:51.162323 29016 log.cc:1196] T e37a9107235b4278bbc17d22e66fc12a P af4206cecdfe47499a68aa849dc33d75: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-1/wal/wals/e37a9107235b4278bbc17d22e66fc12a
I20260928 14:46:51.162590 29016 ts_tablet_manager.cc:1939] T e37a9107235b4278bbc17d22e66fc12a P af4206cecdfe47499a68aa849dc33d75: Deleting consensus metadata
I20260928 14:46:51.163051 29015 ts_tablet_manager.cc:1905] T e37a9107235b4278bbc17d22e66fc12a P db3d7694d02c4b1aae45985c889ae3fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:46:51.163209 28502 catalog_manager.cc:4841] TS af4206cecdfe47499a68aa849dc33d75 (127.9.33.66:42449): tablet e37a9107235b4278bbc17d22e66fc12a (table test-table [id=b502409877744b29a29635e53a5b2c42]) successfully deleted
I20260928 14:46:51.165007 29015 ts_tablet_manager.cc:1918] T e37a9107235b4278bbc17d22e66fc12a P db3d7694d02c4b1aae45985c889ae3fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:46:51.165091 29015 log.cc:1196] T e37a9107235b4278bbc17d22e66fc12a P db3d7694d02c4b1aae45985c889ae3fa: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790606561545179-9349-0/minicluster-data/ts-2/wal/wals/e37a9107235b4278bbc17d22e66fc12a
I20260928 14:46:51.165360 29015 ts_tablet_manager.cc:1939] T e37a9107235b4278bbc17d22e66fc12a P db3d7694d02c4b1aae45985c889ae3fa: Deleting consensus metadata
W20260928 14:46:51.165668 28500 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.33.65:41581 (error 108)
I20260928 14:46:51.165975 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 28712
I20260928 14:46:51.168339 28500 catalog_manager.cc:4841] TS db3d7694d02c4b1aae45985c889ae3fa (127.9.33.67:43669): tablet e37a9107235b4278bbc17d22e66fc12a (table test-table [id=b502409877744b29a29635e53a5b2c42]) successfully deleted
I20260928 14:46:51.168460 28986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.168289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:41581 (local address 127.0.0.1:51424)
0928 14:46:51.168369 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:51.168418 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.65:41581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260928 14:46:51.168613 28500 catalog_manager.cc:4584] TS 09558b0cca6a463889416317eefdd420 (127.9.33.65:41581): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e37a9107235b4278bbc17d22e66fc12a: Network error: Client connection negotiation failed: client connection to 127.9.33.65:41581: connect: Connection refused (error 111)
I20260928 14:46:51.171821 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 28846
I20260928 14:46:51.176472 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 28484
2026-09-28T14:46:51Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1978 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-28T14:46:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:46:51Z Disabled control of system clock
I20260928 14:46:51.204003 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:35117
--webserver_interface=127.9.33.126
--webserver_port=0
--builtin_ntp_servers=127.9.33.84:46029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:35117
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 14:46:51.285663 29021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:46:51.285895 29021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:46:51.285917 29021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:46:51.287070 29021 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:46:51.287110 29021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:46:51.287123 29021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:46:51.287137 29021 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:46:51.287149 29021 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:46:51.288659 29021 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:46029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:35117
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:35117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:46:51.288889 29021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:46:51.289156 29021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:46:51.294255 29029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:51.294220 29026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:51.294236 29027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:51.294622 29021 server_base.cc:1034] running on GCE node
I20260928 14:46:51.294862 29021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:46:51.295151 29021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:46:51.296324 29021 hybrid_clock.cc:648] HybridClock initialized: now 1790606811296305 us; error 28 us; skew 500 ppm
I20260928 14:46:51.297740 29021 webserver.cc:460] Webserver started at http://127.9.33.126:39099/ using document root <none> and password file <none>
I20260928 14:46:51.298043 29021 fs_manager.cc:359] Metadata directory not provided
I20260928 14:46:51.298115 29021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:46:51.298282 29021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:46:51.299433 29021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "529f7a91462d4e5a9e717249ac3dcbff"
format_stamp: "Formatted at 2026-09-28 14:46:51 on dist-test-slave-pv0w"
server_key: "19a91b109401292330f3045b592d73c6"
server_key_iv: "7b958d346b4f6bc434878c7b4c44654f"
server_key_version: "encryptionkey@0"
I20260928 14:46:51.299853 29021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "529f7a91462d4e5a9e717249ac3dcbff"
format_stamp: "Formatted at 2026-09-28 14:46:51 on dist-test-slave-pv0w"
server_key: "19a91b109401292330f3045b592d73c6"
server_key_iv: "7b958d346b4f6bc434878c7b4c44654f"
server_key_version: "encryptionkey@0"
I20260928 14:46:51.301275 29021 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:46:51.302181 29035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:51.302402 29021 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:46:51.302505 29021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "529f7a91462d4e5a9e717249ac3dcbff"
format_stamp: "Formatted at 2026-09-28 14:46:51 on dist-test-slave-pv0w"
server_key: "19a91b109401292330f3045b592d73c6"
server_key_iv: "7b958d346b4f6bc434878c7b4c44654f"
server_key_version: "encryptionkey@0"
I20260928 14:46:51.302591 29021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:46:51.345269 29021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:46:51.345650 29021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:46:51.345814 29021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:46:51.350031 29021 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:35117
I20260928 14:46:51.350067 29087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:35117 every 8 connection(s)
I20260928 14:46:51.350426 29021 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:46:51.351435 29088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:46:51.353883 29088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 529f7a91462d4e5a9e717249ac3dcbff: Bootstrap starting.
I20260928 14:46:51.354502 29088 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 529f7a91462d4e5a9e717249ac3dcbff: Neither blocks nor log segments found. Creating new log.
I20260928 14:46:51.354823 29088 log.cc:824] T 00000000000000000000000000000000 P 529f7a91462d4e5a9e717249ac3dcbff: Log is configured to *not* fsync() on all Append() calls
I20260928 14:46:51.355533 29088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 529f7a91462d4e5a9e717249ac3dcbff: No bootstrap required, opened a new log
I20260928 14:46:51.357321 29088 raft_consensus.cc:348] T 00000000000000000000000000000000 P 529f7a91462d4e5a9e717249ac3dcbff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "529f7a91462d4e5a9e717249ac3dcbff" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35117 } }
I20260928 14:46:51.357461 29088 raft_consensus.cc:374] T 00000000000000000000000000000000 P 529f7a91462d4e5a9e717249ac3dcbff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:46:51.357506 29088 raft_consensus.cc:729] T 00000000000000000000000000000000 P 529f7a91462d4e5a9e717249ac3dcbff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 529f7a91462d4e5a9e717249ac3dcbff, State: Initialized, Role: FOLLOWER
I20260928 14:46:51.357666 29088 consensus_queue.cc:260] T 00000000000000000000000000000000 P 529f7a91462d4e5a9e717249ac3dcbff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "529f7a91462d4e5a9e717249ac3dcbff" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35117 } }
I20260928 14:46:51.357728 29088 raft_consensus.cc:388] T 00000000000000000000000000000000 P 529f7a91462d4e5a9e717249ac3dcbff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:46:51.357767 29088 raft_consensus.cc:482] T 00000000000000000000000000000000 P 529f7a91462d4e5a9e717249ac3dcbff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:46:51.357816 29088 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 529f7a91462d4e5a9e717249ac3dcbff [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:46:51.358352 29088 raft_consensus.cc:504] T 00000000000000000000000000000000 P 529f7a91462d4e5a9e717249ac3dcbff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "529f7a91462d4e5a9e717249ac3dcbff" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35117 } }
I20260928 14:46:51.358467 29088 leader_election.cc:302] T 00000000000000000000000000000000 P 529f7a91462d4e5a9e717249ac3dcbff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 529f7a91462d4e5a9e717249ac3dcbff; no voters:
I20260928 14:46:51.358655 29088 leader_election.cc:288] T 00000000000000000000000000000000 P 529f7a91462d4e5a9e717249ac3dcbff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:46:51.358704 29091 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 529f7a91462d4e5a9e717249ac3dcbff [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:46:51.358906 29091 raft_consensus.cc:686] T 00000000000000000000000000000000 P 529f7a91462d4e5a9e717249ac3dcbff [term 1 LEADER]: Becoming Leader. State: Replica: 529f7a91462d4e5a9e717249ac3dcbff, State: Running, Role: LEADER
I20260928 14:46:51.358940 29088 sys_catalog.cc:564] T 00000000000000000000000000000000 P 529f7a91462d4e5a9e717249ac3dcbff [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:46:51.358943 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 29021
I20260928 14:46:51.359041 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:46:51.359014 29091 consensus_queue.cc:237] T 00000000000000000000000000000000 P 529f7a91462d4e5a9e717249ac3dcbff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "529f7a91462d4e5a9e717249ac3dcbff" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35117 } }
I20260928 14:46:51.359268 9349 external_mini_cluster.cc:1428] Setting key 3383313abe2b03091ad92e71730759ec
I20260928 14:46:51.359427 29092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 529f7a91462d4e5a9e717249ac3dcbff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "529f7a91462d4e5a9e717249ac3dcbff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "529f7a91462d4e5a9e717249ac3dcbff" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35117 } } }
I20260928 14:46:51.359524 29092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 529f7a91462d4e5a9e717249ac3dcbff [sys.catalog]: This master's current role is: LEADER
I20260928 14:46:51.359802 29093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 529f7a91462d4e5a9e717249ac3dcbff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 529f7a91462d4e5a9e717249ac3dcbff. Latest consensus state: current_term: 1 leader_uuid: "529f7a91462d4e5a9e717249ac3dcbff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "529f7a91462d4e5a9e717249ac3dcbff" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35117 } } }
I20260928 14:46:51.359932 29093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 529f7a91462d4e5a9e717249ac3dcbff [sys.catalog]: This master's current role is: LEADER
I20260928 14:46:51.360081 29099 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:46:51.361054 29099 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:46:51.363068 29099 catalog_manager.cc:1347] Generated new cluster ID: 864c5509b6214023973c02cd970cfad4
I20260928 14:46:51.363122 29099 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:46:51.365161 29102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.359771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40504 (local address 127.9.33.126:35117)
0928 14:46:51.360035 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:51.360045 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:51.360079 (+ 34us) server_negotiation.cc:407] Connection header received
0928 14:46:51.360224 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:51.360232 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:51.360305 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:51.360462 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:51.360727 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.361971 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.362922 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.363534 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.363655 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:51.364425 (+ 770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:51.364429 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:51.364430 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:46:51.364658 (+ 228us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:51.364701 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:46:51.364716 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:51.364791 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:51.364860 (+ 69us) server_negotiation.cc:299] Negotiation successful
0928 14:46:51.364942 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":81,"threads_started":1}
I20260928 14:46:51.382835 29099 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:46:51.383587 29099 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:46:51.397948 29099 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 529f7a91462d4e5a9e717249ac3dcbff: Generated new TSK 0
I20260928 14:46:51.398208 29099 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:46:51.403667 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.65:0
--local_ip_for_outbound_sockets=127.9.33.65
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:35117
--builtin_ntp_servers=127.9.33.84:46029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260928 14:46:51.486761 29112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:46:51.487003 29112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:46:51.487043 29112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:46:51.488334 29112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:46:51.488389 29112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:46:51.488431 29112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.65
I20260928 14:46:51.489804 29112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:46029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.33.65
--webserver_port=0
--tserver_master_addrs=127.9.33.126:35117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.65
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:46:51.490058 29112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:46:51.490346 29112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:46:51.495342 29120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:51.495325 29118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:51.495393 29112 server_base.cc:1034] running on GCE node
W20260928 14:46:51.495326 29117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:51.495770 29112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:46:51.495995 29112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:46:51.497165 29112 hybrid_clock.cc:648] HybridClock initialized: now 1790606811497152 us; error 28 us; skew 500 ppm
I20260928 14:46:51.498690 29112 webserver.cc:460] Webserver started at http://127.9.33.65:33351/ using document root <none> and password file <none>
I20260928 14:46:51.498891 29112 fs_manager.cc:359] Metadata directory not provided
I20260928 14:46:51.498941 29112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:46:51.499038 29112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:46:51.500162 29112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-0/data/instance:
uuid: "70e2936002754f0b93399ecaacc2db2e"
format_stamp: "Formatted at 2026-09-28 14:46:51 on dist-test-slave-pv0w"
server_key: "ff75ffd5fb5f5fbe1d0e2766a153e061"
server_key_iv: "b322e9a2f23459b881482347cba683d1"
server_key_version: "encryptionkey@0"
I20260928 14:46:51.500480 29112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance:
uuid: "70e2936002754f0b93399ecaacc2db2e"
format_stamp: "Formatted at 2026-09-28 14:46:51 on dist-test-slave-pv0w"
server_key: "ff75ffd5fb5f5fbe1d0e2766a153e061"
server_key_iv: "b322e9a2f23459b881482347cba683d1"
server_key_version: "encryptionkey@0"
I20260928 14:46:51.501678 29112 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:46:51.502360 29126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:51.502542 29112 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:51.502609 29112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-0/wal
uuid: "70e2936002754f0b93399ecaacc2db2e"
format_stamp: "Formatted at 2026-09-28 14:46:51 on dist-test-slave-pv0w"
server_key: "ff75ffd5fb5f5fbe1d0e2766a153e061"
server_key_iv: "b322e9a2f23459b881482347cba683d1"
server_key_version: "encryptionkey@0"
I20260928 14:46:51.502660 29112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:46:51.527050 29112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:46:51.527339 29112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:46:51.527447 29112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:46:51.527673 29112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:46:51.528090 29112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:46:51.528126 29112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:51.528148 29112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:46:51.528195 29112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:51.533845 29112 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.65:41237
I20260928 14:46:51.533946 29239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.65:41237 every 8 connection(s)
I20260928 14:46:51.534261 29112 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-0/data/info.pb
I20260928 14:46:51.538951 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 29112
I20260928 14:46:51.539067 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-0/wal/instance
I20260928 14:46:51.539279 9349 external_mini_cluster.cc:1428] Setting key d55fd5ffd175759437240d4c8b79ca4b
I20260928 14:46:51.539621 29102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.535057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:37515 (local address 127.9.33.126:35117)
0928 14:46:51.535121 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:51.535123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:51.535374 (+ 251us) server_negotiation.cc:407] Connection header received
0928 14:46:51.535905 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:51.535906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:51.535917 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:51.535985 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:51.536690 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.537207 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.538153 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.538347 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.538403 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:51.539255 (+ 852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:51.539257 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:51.539257 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:46:51.539262 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:51.539276 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:46:51.539280 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:51.539330 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:51.539449 (+ 119us) server_negotiation.cc:299] Negotiation successful
0928 14:46:51.539506 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260928 14:46:51.539855 29242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.535035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:35117 (local address 127.9.33.65:37515)
0928 14:46:51.535248 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:51.535290 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:51.535793 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:51.536064 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:51.536067 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:51.536076 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:46:51.536606 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:51.536610 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.537285 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.537286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:51.538076 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:51.538079 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.538420 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.538421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:51.538530 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:51.539073 (+ 543us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:46:51.539073 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:46:51.539116 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:46:51.539167 (+ 51us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:46:51.539347 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:46:51.539354 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:46:51.539358 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:46:51.539437 (+ 79us) client_negotiation.cc:240] Negotiation successful
0928 14:46:51.539558 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":89,"threads_started":1}
I20260928 14:46:51.540362 29240 heartbeater.cc:347] Connected to a master server at 127.9.33.126:35117
I20260928 14:46:51.540490 29240 heartbeater.cc:464] Registering TS with master...
I20260928 14:46:51.540475 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.66:0
--local_ip_for_outbound_sockets=127.9.33.66
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:35117
--builtin_ntp_servers=127.9.33.84:46029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 14:46:51.540719 29240 heartbeater.cc:511] Master 127.9.33.126:35117 requested a full tablet report, sending...
I20260928 14:46:51.541256 29052 ts_manager.cc:194] Registered new tserver with Master: 70e2936002754f0b93399ecaacc2db2e (127.9.33.65:41237)
I20260928 14:46:51.541869 29052 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.33.65:37515
W20260928 14:46:51.622699 29243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:46:51.622910 29243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:46:51.622931 29243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:46:51.624190 29243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:46:51.624233 29243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:46:51.624259 29243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.66
I20260928 14:46:51.625705 29243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:46029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.33.66
--webserver_port=0
--tserver_master_addrs=127.9.33.126:35117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.66
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:46:51.625960 29243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:46:51.626267 29243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:46:51.631331 29248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:51.631331 29249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:51.631323 29251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:51.631524 29243 server_base.cc:1034] running on GCE node
I20260928 14:46:51.631870 29243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:46:51.632184 29243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:46:51.633375 29243 hybrid_clock.cc:648] HybridClock initialized: now 1790606811633367 us; error 37 us; skew 500 ppm
I20260928 14:46:51.634837 29243 webserver.cc:460] Webserver started at http://127.9.33.66:42163/ using document root <none> and password file <none>
I20260928 14:46:51.635113 29243 fs_manager.cc:359] Metadata directory not provided
I20260928 14:46:51.635186 29243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:46:51.635340 29243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:46:51.636524 29243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-1/data/instance:
uuid: "5b484d024a104088a9cb1a0f34d2a58d"
format_stamp: "Formatted at 2026-09-28 14:46:51 on dist-test-slave-pv0w"
server_key: "f7b236e53bc1394b5e7881ee1cb2ec05"
server_key_iv: "f66471fb1902c5004d1f9f6d6c3613df"
server_key_version: "encryptionkey@0"
I20260928 14:46:51.636889 29243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance:
uuid: "5b484d024a104088a9cb1a0f34d2a58d"
format_stamp: "Formatted at 2026-09-28 14:46:51 on dist-test-slave-pv0w"
server_key: "f7b236e53bc1394b5e7881ee1cb2ec05"
server_key_iv: "f66471fb1902c5004d1f9f6d6c3613df"
server_key_version: "encryptionkey@0"
I20260928 14:46:51.638228 29243 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:46:51.638955 29257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:51.639125 29243 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:46:51.639227 29243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-1/wal
uuid: "5b484d024a104088a9cb1a0f34d2a58d"
format_stamp: "Formatted at 2026-09-28 14:46:51 on dist-test-slave-pv0w"
server_key: "f7b236e53bc1394b5e7881ee1cb2ec05"
server_key_iv: "f66471fb1902c5004d1f9f6d6c3613df"
server_key_version: "encryptionkey@0"
I20260928 14:46:51.639302 29243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:46:51.653090 29243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:46:51.653481 29243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:46:51.653638 29243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:46:51.653921 29243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:46:51.654363 29243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:46:51.654409 29243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:51.654452 29243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:46:51.654475 29243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:51.662180 29243 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.66:34365
I20260928 14:46:51.662221 29370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.66:34365 every 8 connection(s)
I20260928 14:46:51.662599 29243 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-1/data/info.pb
I20260928 14:46:51.665639 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 29243
I20260928 14:46:51.665776 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-1/wal/instance
I20260928 14:46:51.666002 9349 external_mini_cluster.cc:1428] Setting key dd981ccf11eb13617452abc43698c62f
I20260928 14:46:51.667122 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.33.67:0
--local_ip_for_outbound_sockets=127.9.33.67
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:35117
--builtin_ntp_servers=127.9.33.84:46029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260928 14:46:51.668344 29102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.663395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.66:33775 (local address 127.9.33.126:35117)
0928 14:46:51.663443 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:51.663446 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:51.663680 (+ 234us) server_negotiation.cc:407] Connection header received
0928 14:46:51.664264 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:51.664265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:51.664279 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:51.664332 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:51.665019 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.665509 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.666769 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.667058 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.667132 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:51.668079 (+ 947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:51.668080 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:51.668080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:46:51.668085 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:51.668100 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:46:51.668104 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:51.668147 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:51.668233 (+ 86us) server_negotiation.cc:299] Negotiation successful
0928 14:46:51.668266 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 14:46:51.668576 29373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.663394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:35117 (local address 127.9.33.66:33775)
0928 14:46:51.663575 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:51.663608 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:51.664170 (+ 562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:51.664386 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:51.664389 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:51.664398 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:46:51.664922 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:51.664926 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.665625 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.665626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:51.666674 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:51.666677 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.667141 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.667143 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:51.667306 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:51.667926 (+ 620us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:46:51.667927 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:46:51.667956 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:46:51.667979 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:46:51.668158 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:46:51.668162 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:46:51.668166 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:46:51.668231 (+ 65us) client_negotiation.cc:240] Negotiation successful
0928 14:46:51.668334 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":89,"threads_started":1}
I20260928 14:46:51.669034 29371 heartbeater.cc:347] Connected to a master server at 127.9.33.126:35117
I20260928 14:46:51.669137 29371 heartbeater.cc:464] Registering TS with master...
I20260928 14:46:51.669312 29371 heartbeater.cc:511] Master 127.9.33.126:35117 requested a full tablet report, sending...
I20260928 14:46:51.669708 29052 ts_manager.cc:194] Registered new tserver with Master: 5b484d024a104088a9cb1a0f34d2a58d (127.9.33.66:34365)
I20260928 14:46:51.670097 29052 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.33.66:33775
W20260928 14:46:51.749017 29374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:46:51.749272 29374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:46:51.749310 29374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:46:51.750471 29374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:46:51.750535 29374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:46:51.750578 29374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.33.67
I20260928 14:46:51.752071 29374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:46029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.33.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.33.67
--webserver_port=0
--tserver_master_addrs=127.9.33.126:35117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.33.67
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:46:51.752348 29374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:46:51.752693 29374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:46:51.758204 29379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:51.758277 29374 server_base.cc:1034] running on GCE node
W20260928 14:46:51.758198 29380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:51.758265 29382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:51.758771 29374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:46:51.759059 29374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:46:51.760241 29374 hybrid_clock.cc:648] HybridClock initialized: now 1790606811760221 us; error 46 us; skew 500 ppm
I20260928 14:46:51.761945 29374 webserver.cc:460] Webserver started at http://127.9.33.67:37167/ using document root <none> and password file <none>
I20260928 14:46:51.762284 29374 fs_manager.cc:359] Metadata directory not provided
I20260928 14:46:51.762355 29374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:46:51.762506 29374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:46:51.763758 29374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-2/data/instance:
uuid: "073ca38165d84135a2e04c45a80fd517"
format_stamp: "Formatted at 2026-09-28 14:46:51 on dist-test-slave-pv0w"
server_key: "cc8d6ee7416eec86cf8f5c4e37f2a250"
server_key_iv: "aff2f5cd35880a0bd17a611c4ef3dfa0"
server_key_version: "encryptionkey@0"
I20260928 14:46:51.764176 29374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance:
uuid: "073ca38165d84135a2e04c45a80fd517"
format_stamp: "Formatted at 2026-09-28 14:46:51 on dist-test-slave-pv0w"
server_key: "cc8d6ee7416eec86cf8f5c4e37f2a250"
server_key_iv: "aff2f5cd35880a0bd17a611c4ef3dfa0"
server_key_version: "encryptionkey@0"
I20260928 14:46:51.765496 29374 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:46:51.766170 29388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:51.766345 29374 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:46:51.766435 29374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-2/wal
uuid: "073ca38165d84135a2e04c45a80fd517"
format_stamp: "Formatted at 2026-09-28 14:46:51 on dist-test-slave-pv0w"
server_key: "cc8d6ee7416eec86cf8f5c4e37f2a250"
server_key_iv: "aff2f5cd35880a0bd17a611c4ef3dfa0"
server_key_version: "encryptionkey@0"
I20260928 14:46:51.766508 29374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:46:51.778036 29374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:46:51.778352 29374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:46:51.778517 29374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:46:51.778741 29374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260928 14:46:51.779083 29374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260928 14:46:51.779135 29374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:51.779177 29374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260928 14:46:51.779206 29374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:51.785386 29374 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.67:34517
I20260928 14:46:51.785434 29501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.67:34517 every 8 connection(s)
I20260928 14:46:51.785769 29374 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-2/data/info.pb
I20260928 14:46:51.791009 29102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.786490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:32977 (local address 127.9.33.126:35117)
0928 14:46:51.786551 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:51.786553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:51.786821 (+ 268us) server_negotiation.cc:407] Connection header received
0928 14:46:51.787361 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:51.787362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:51.787375 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:51.787432 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:51.788162 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.788666 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.789640 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.789839 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.789910 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:51.790711 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:51.790712 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:51.790713 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:46:51.790718 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:51.790739 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:46:51.790748 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:51.790805 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:51.790896 (+ 91us) server_negotiation.cc:299] Negotiation successful
0928 14:46:51.790927 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260928 14:46:51.791245 29504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.786486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:35117 (local address 127.9.33.67:32977)
0928 14:46:51.786695 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:51.786730 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:51.787257 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:51.787512 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:51.787515 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:51.787524 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:46:51.788063 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:51.788067 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.788759 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.788760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:51.789543 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:51.789545 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.789928 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.789929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:51.790055 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:51.790562 (+ 507us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0928 14:46:51.790563 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:46:51.790589 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0928 14:46:51.790611 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:46:51.790806 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:46:51.790812 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:46:51.790816 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:46:51.790897 (+ 81us) client_negotiation.cc:240] Negotiation successful
0928 14:46:51.791000 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":91,"threads_started":1}
I20260928 14:46:51.791872 29502 heartbeater.cc:347] Connected to a master server at 127.9.33.126:35117
I20260928 14:46:51.791965 29502 heartbeater.cc:464] Registering TS with master...
I20260928 14:46:51.792152 29502 heartbeater.cc:511] Master 127.9.33.126:35117 requested a full tablet report, sending...
I20260928 14:46:51.792553 29052 ts_manager.cc:194] Registered new tserver with Master: 073ca38165d84135a2e04c45a80fd517 (127.9.33.67:34517)
I20260928 14:46:51.792753 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 29374
I20260928 14:46:51.792865 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-2/wal/instance
I20260928 14:46:51.793068 9349 external_mini_cluster.cc:1428] Setting key e6a744cd6b44c6ace5a576641dd8887a
I20260928 14:46:51.793056 29052 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.33.67:32977
I20260928 14:46:51.794009 9349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260928 14:46:51.798307 29102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.795363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40512 (local address 127.9.33.126:35117)
0928 14:46:51.795413 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:51.795415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:51.795573 (+ 158us) server_negotiation.cc:407] Connection header received
0928 14:46:51.795626 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:51.795627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:51.795637 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:51.795677 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:51.795864 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.796328 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.797118 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.797303 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.797368 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:51.798061 (+ 693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:51.798062 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:51.798063 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:46:51.798067 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:51.798080 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:46:51.798084 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:51.798129 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:51.798183 (+ 54us) server_negotiation.cc:299] Negotiation successful
0928 14:46:51.798206 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 14:46:51.799919 29052 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40512:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260928 14:46:51.801438 29052 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260928 14:46:51.808212 29512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.803626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50832 (local address 127.9.33.65:41237)
0928 14:46:51.803879 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:51.803885 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:51.804052 (+ 167us) server_negotiation.cc:407] Connection header received
0928 14:46:51.804105 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:51.804107 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:51.804124 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:51.804224 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:46:51.804427 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.805617 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.807090 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.807807 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.807888 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:51.807987 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:51.808087 (+ 100us) server_negotiation.cc:299] Negotiation successful
0928 14:46:51.808122 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":138,"threads_started":1}
I20260928 14:46:51.808213 29511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.803499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:34365 (local address 127.0.0.1:32866)
0928 14:46:51.803626 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:51.803667 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:51.803758 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:51.804257 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:51.804260 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:51.804262 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:46:51.804329 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:51.804331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.805718 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.805719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:51.806905 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:51.806908 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.807911 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.807912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:51.807975 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:51.807995 (+ 20us) client_negotiation.cc:769] Sending connection context
0928 14:46:51.808042 (+ 47us) client_negotiation.cc:240] Negotiation successful
0928 14:46:51.808102 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":63,"thread_start_us":71,"threads_started":1}
I20260928 14:46:51.808502 29513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.803508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:41237 (local address 127.0.0.1:50832)
0928 14:46:51.803931 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:51.803942 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:51.803996 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:51.804304 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:51.804304 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:51.804307 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:46:51.804348 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:51.804349 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.805705 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.805706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:51.807009 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:51.807012 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.807893 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.807894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:51.807974 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:51.807989 (+ 15us) client_negotiation.cc:769] Sending connection context
0928 14:46:51.808070 (+ 81us) client_negotiation.cc:240] Negotiation successful
0928 14:46:51.808111 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"mutex_wait_us":54,"thread_start_us":55,"threads_started":1}
I20260928 14:46:51.808568 29515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.803880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32866 (local address 127.9.33.66:34365)
0928 14:46:51.804080 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:51.804084 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:51.804093 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:46:51.804129 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:51.804130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:51.804144 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:51.804414 (+ 270us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:46:51.804477 (+ 63us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.805649 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.806993 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.807806 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.808176 (+ 370us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:51.808278 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:51.808405 (+ 127us) server_negotiation.cc:299] Negotiation successful
0928 14:46:51.808472 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":91,"threads_started":1}
I20260928 14:46:51.809250 29174 tablet_service.cc:1467] Processing CreateTablet for tablet fc0171a552ef47a38b936bd1b970966b (DEFAULT_TABLE table=test-table [id=c9cc09a34da7420fb867cb407fd94900]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:46:51.809247 29305 tablet_service.cc:1467] Processing CreateTablet for tablet fc0171a552ef47a38b936bd1b970966b (DEFAULT_TABLE table=test-table [id=c9cc09a34da7420fb867cb407fd94900]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:46:51.809623 29174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc0171a552ef47a38b936bd1b970966b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:46:51.809623 29305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc0171a552ef47a38b936bd1b970966b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:46:51.810103 29510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.803705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:34517 (local address 127.0.0.1:40580)
0928 14:46:51.805760 (+ 2055us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:51.805769 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:51.805846 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:51.806333 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:51.806335 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:51.806337 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:46:51.806404 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:51.806405 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.807502 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.807503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:51.809086 (+ 1583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:51.809088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.809872 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.809873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:51.809933 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:51.809937 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:46:51.809993 (+ 56us) client_negotiation.cc:240] Negotiation successful
0928 14:46:51.810034 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2041,"thread_start_us":87,"threads_started":1}
I20260928 14:46:51.810119 29514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.803827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40580 (local address 127.9.33.67:34517)
0928 14:46:51.804008 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:51.804013 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:51.805851 (+ 1838us) server_negotiation.cc:407] Connection header received
0928 14:46:51.805939 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:51.805942 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:51.805955 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:51.806044 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:46:51.806467 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.807424 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.809173 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.809788 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.809871 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:51.809955 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:51.810007 (+ 52us) server_negotiation.cc:299] Negotiation successful
0928 14:46:51.810041 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":108,"threads_started":1}
I20260928 14:46:51.810985 29436 tablet_service.cc:1467] Processing CreateTablet for tablet fc0171a552ef47a38b936bd1b970966b (DEFAULT_TABLE table=test-table [id=c9cc09a34da7420fb867cb407fd94900]), partition=RANGE (key) PARTITION UNBOUNDED
I20260928 14:46:51.811295 29436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc0171a552ef47a38b936bd1b970966b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:46:51.813151 29516 tablet_bootstrap.cc:492] T fc0171a552ef47a38b936bd1b970966b P 70e2936002754f0b93399ecaacc2db2e: Bootstrap starting.
I20260928 14:46:51.813905 29516 tablet_bootstrap.cc:654] T fc0171a552ef47a38b936bd1b970966b P 70e2936002754f0b93399ecaacc2db2e: Neither blocks nor log segments found. Creating new log.
I20260928 14:46:51.814185 29518 tablet_bootstrap.cc:492] T fc0171a552ef47a38b936bd1b970966b P 073ca38165d84135a2e04c45a80fd517: Bootstrap starting.
I20260928 14:46:51.814316 29517 tablet_bootstrap.cc:492] T fc0171a552ef47a38b936bd1b970966b P 5b484d024a104088a9cb1a0f34d2a58d: Bootstrap starting.
I20260928 14:46:51.814361 29516 log.cc:824] T fc0171a552ef47a38b936bd1b970966b P 70e2936002754f0b93399ecaacc2db2e: Log is configured to *not* fsync() on all Append() calls
I20260928 14:46:51.814919 29517 tablet_bootstrap.cc:654] T fc0171a552ef47a38b936bd1b970966b P 5b484d024a104088a9cb1a0f34d2a58d: Neither blocks nor log segments found. Creating new log.
I20260928 14:46:51.814941 29518 tablet_bootstrap.cc:654] T fc0171a552ef47a38b936bd1b970966b P 073ca38165d84135a2e04c45a80fd517: Neither blocks nor log segments found. Creating new log.
I20260928 14:46:51.815228 29518 log.cc:824] T fc0171a552ef47a38b936bd1b970966b P 073ca38165d84135a2e04c45a80fd517: Log is configured to *not* fsync() on all Append() calls
I20260928 14:46:51.815241 29516 tablet_bootstrap.cc:492] T fc0171a552ef47a38b936bd1b970966b P 70e2936002754f0b93399ecaacc2db2e: No bootstrap required, opened a new log
I20260928 14:46:51.815239 29517 log.cc:824] T fc0171a552ef47a38b936bd1b970966b P 5b484d024a104088a9cb1a0f34d2a58d: Log is configured to *not* fsync() on all Append() calls
I20260928 14:46:51.815317 29516 ts_tablet_manager.cc:1397] T fc0171a552ef47a38b936bd1b970966b P 70e2936002754f0b93399ecaacc2db2e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:46:51.816092 29518 tablet_bootstrap.cc:492] T fc0171a552ef47a38b936bd1b970966b P 073ca38165d84135a2e04c45a80fd517: No bootstrap required, opened a new log
I20260928 14:46:51.816159 29518 ts_tablet_manager.cc:1397] T fc0171a552ef47a38b936bd1b970966b P 073ca38165d84135a2e04c45a80fd517: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:46:51.816340 29517 tablet_bootstrap.cc:492] T fc0171a552ef47a38b936bd1b970966b P 5b484d024a104088a9cb1a0f34d2a58d: No bootstrap required, opened a new log
I20260928 14:46:51.816430 29517 ts_tablet_manager.cc:1397] T fc0171a552ef47a38b936bd1b970966b P 5b484d024a104088a9cb1a0f34d2a58d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260928 14:46:51.818056 29518 raft_consensus.cc:348] T fc0171a552ef47a38b936bd1b970966b P 073ca38165d84135a2e04c45a80fd517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e2936002754f0b93399ecaacc2db2e" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41237 } } peers { permanent_uuid: "073ca38165d84135a2e04c45a80fd517" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34517 } } peers { permanent_uuid: "5b484d024a104088a9cb1a0f34d2a58d" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 34365 } }
I20260928 14:46:51.818059 29516 raft_consensus.cc:348] T fc0171a552ef47a38b936bd1b970966b P 70e2936002754f0b93399ecaacc2db2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e2936002754f0b93399ecaacc2db2e" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41237 } } peers { permanent_uuid: "073ca38165d84135a2e04c45a80fd517" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34517 } } peers { permanent_uuid: "5b484d024a104088a9cb1a0f34d2a58d" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 34365 } }
I20260928 14:46:51.818221 29518 raft_consensus.cc:374] T fc0171a552ef47a38b936bd1b970966b P 073ca38165d84135a2e04c45a80fd517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:46:51.818222 29516 raft_consensus.cc:374] T fc0171a552ef47a38b936bd1b970966b P 70e2936002754f0b93399ecaacc2db2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:46:51.818265 29516 raft_consensus.cc:729] T fc0171a552ef47a38b936bd1b970966b P 70e2936002754f0b93399ecaacc2db2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70e2936002754f0b93399ecaacc2db2e, State: Initialized, Role: FOLLOWER
I20260928 14:46:51.818265 29518 raft_consensus.cc:729] T fc0171a552ef47a38b936bd1b970966b P 073ca38165d84135a2e04c45a80fd517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 073ca38165d84135a2e04c45a80fd517, State: Initialized, Role: FOLLOWER
I20260928 14:46:51.818400 29518 consensus_queue.cc:260] T fc0171a552ef47a38b936bd1b970966b P 073ca38165d84135a2e04c45a80fd517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e2936002754f0b93399ecaacc2db2e" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41237 } } peers { permanent_uuid: "073ca38165d84135a2e04c45a80fd517" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34517 } } peers { permanent_uuid: "5b484d024a104088a9cb1a0f34d2a58d" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 34365 } }
I20260928 14:46:51.818401 29516 consensus_queue.cc:260] T fc0171a552ef47a38b936bd1b970966b P 70e2936002754f0b93399ecaacc2db2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e2936002754f0b93399ecaacc2db2e" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41237 } } peers { permanent_uuid: "073ca38165d84135a2e04c45a80fd517" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34517 } } peers { permanent_uuid: "5b484d024a104088a9cb1a0f34d2a58d" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 34365 } }
I20260928 14:46:51.818686 29516 ts_tablet_manager.cc:1428] T fc0171a552ef47a38b936bd1b970966b P 70e2936002754f0b93399ecaacc2db2e: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260928 14:46:51.818681 29518 ts_tablet_manager.cc:1428] T fc0171a552ef47a38b936bd1b970966b P 073ca38165d84135a2e04c45a80fd517: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260928 14:46:51.818869 29502 heartbeater.cc:503] Master 127.9.33.126:35117 was elected leader, sending a full tablet report...
I20260928 14:46:51.818892 29240 heartbeater.cc:503] Master 127.9.33.126:35117 was elected leader, sending a full tablet report...
I20260928 14:46:51.819029 29517 raft_consensus.cc:348] T fc0171a552ef47a38b936bd1b970966b P 5b484d024a104088a9cb1a0f34d2a58d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e2936002754f0b93399ecaacc2db2e" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41237 } } peers { permanent_uuid: "073ca38165d84135a2e04c45a80fd517" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34517 } } peers { permanent_uuid: "5b484d024a104088a9cb1a0f34d2a58d" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 34365 } }
I20260928 14:46:51.819200 29517 raft_consensus.cc:374] T fc0171a552ef47a38b936bd1b970966b P 5b484d024a104088a9cb1a0f34d2a58d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:46:51.819247 29517 raft_consensus.cc:729] T fc0171a552ef47a38b936bd1b970966b P 5b484d024a104088a9cb1a0f34d2a58d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b484d024a104088a9cb1a0f34d2a58d, State: Initialized, Role: FOLLOWER
I20260928 14:46:51.819370 29517 consensus_queue.cc:260] T fc0171a552ef47a38b936bd1b970966b P 5b484d024a104088a9cb1a0f34d2a58d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e2936002754f0b93399ecaacc2db2e" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41237 } } peers { permanent_uuid: "073ca38165d84135a2e04c45a80fd517" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34517 } } peers { permanent_uuid: "5b484d024a104088a9cb1a0f34d2a58d" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 34365 } }
I20260928 14:46:51.819612 29517 ts_tablet_manager.cc:1428] T fc0171a552ef47a38b936bd1b970966b P 5b484d024a104088a9cb1a0f34d2a58d: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260928 14:46:51.819765 29371 heartbeater.cc:503] Master 127.9.33.126:35117 was elected leader, sending a full tablet report...
I20260928 14:46:51.853641 29522 raft_consensus.cc:482] T fc0171a552ef47a38b936bd1b970966b P 70e2936002754f0b93399ecaacc2db2e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:46:51.853823 29522 raft_consensus.cc:504] T fc0171a552ef47a38b936bd1b970966b P 70e2936002754f0b93399ecaacc2db2e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e2936002754f0b93399ecaacc2db2e" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41237 } } peers { permanent_uuid: "073ca38165d84135a2e04c45a80fd517" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34517 } } peers { permanent_uuid: "5b484d024a104088a9cb1a0f34d2a58d" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 34365 } }
I20260928 14:46:51.854262 29522 leader_election.cc:288] T fc0171a552ef47a38b936bd1b970966b P 70e2936002754f0b93399ecaacc2db2e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 073ca38165d84135a2e04c45a80fd517 (127.9.33.67:34517), 5b484d024a104088a9cb1a0f34d2a58d (127.9.33.66:34365)
I20260928 14:46:51.857167 29514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.854481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:54897 (local address 127.9.33.67:34517)
0928 14:46:51.854605 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:51.854607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:51.854619 (+ 12us) server_negotiation.cc:407] Connection header received
0928 14:46:51.854703 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:51.854703 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:51.854715 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:51.854759 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:46:51.854914 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.855399 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.856436 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.856816 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.856897 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:51.856908 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:51.857060 (+ 152us) server_negotiation.cc:299] Negotiation successful
0928 14:46:51.857099 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260928 14:46:51.857167 29242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.854398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.67:34517 (local address 127.9.33.65:54897)
0928 14:46:51.854573 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:51.854586 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:51.854636 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:51.854761 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:51.854761 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:51.854763 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:46:51.854841 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:51.854842 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.855496 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.855497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:51.856355 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:51.856357 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.856895 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.856895 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:51.856965 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:51.856968 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:46:51.857052 (+ 84us) client_negotiation.cc:240] Negotiation successful
0928 14:46:51.857096 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152}
I20260928 14:46:51.857506 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc0171a552ef47a38b936bd1b970966b" candidate_uuid: "70e2936002754f0b93399ecaacc2db2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "073ca38165d84135a2e04c45a80fd517" is_pre_election: true
I20260928 14:46:51.857549 29523 raft_consensus.cc:482] T fc0171a552ef47a38b936bd1b970966b P 073ca38165d84135a2e04c45a80fd517 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260928 14:46:51.857673 29523 raft_consensus.cc:504] T fc0171a552ef47a38b936bd1b970966b P 073ca38165d84135a2e04c45a80fd517 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e2936002754f0b93399ecaacc2db2e" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41237 } } peers { permanent_uuid: "073ca38165d84135a2e04c45a80fd517" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34517 } } peers { permanent_uuid: "5b484d024a104088a9cb1a0f34d2a58d" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 34365 } }
I20260928 14:46:51.857826 29456 raft_consensus.cc:2454] T fc0171a552ef47a38b936bd1b970966b P 073ca38165d84135a2e04c45a80fd517 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70e2936002754f0b93399ecaacc2db2e in term 0.
I20260928 14:46:51.857934 29515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.854587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.65:60401 (local address 127.9.33.66:34365)
0928 14:46:51.854641 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:51.854643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:51.854749 (+ 106us) server_negotiation.cc:407] Connection header received
0928 14:46:51.854829 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:51.854830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:51.854844 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:51.854901 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:46:51.855063 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.855860 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.856797 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.857156 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.857237 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:51.857246 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:51.857815 (+ 569us) server_negotiation.cc:299] Negotiation successful
0928 14:46:51.857872 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260928 14:46:51.858090 29130 leader_election.cc:302] T fc0171a552ef47a38b936bd1b970966b P 70e2936002754f0b93399ecaacc2db2e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 073ca38165d84135a2e04c45a80fd517, 70e2936002754f0b93399ecaacc2db2e; no voters:
I20260928 14:46:51.858206 29522 raft_consensus.cc:2781] T fc0171a552ef47a38b936bd1b970966b P 70e2936002754f0b93399ecaacc2db2e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260928 14:46:51.858237 29525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.854448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:34365 (local address 127.9.33.65:60401)
0928 14:46:51.854664 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:51.854675 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:51.854761 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:51.854904 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:51.854905 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:51.854907 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:46:51.854984 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:51.854992 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.855959 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.855960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:51.856706 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:51.856708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.857244 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.857245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:51.857315 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:51.857319 (+ 4us) client_negotiation.cc:769] Sending connection context
0928 14:46:51.857957 (+ 638us) client_negotiation.cc:240] Negotiation successful
0928 14:46:51.858146 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":121,"threads_started":1}
I20260928 14:46:51.858269 29522 raft_consensus.cc:482] T fc0171a552ef47a38b936bd1b970966b P 70e2936002754f0b93399ecaacc2db2e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260928 14:46:51.858297 29522 raft_consensus.cc:3037] T fc0171a552ef47a38b936bd1b970966b P 70e2936002754f0b93399ecaacc2db2e [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:46:51.858412 29523 leader_election.cc:288] T fc0171a552ef47a38b936bd1b970966b P 073ca38165d84135a2e04c45a80fd517 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 70e2936002754f0b93399ecaacc2db2e (127.9.33.65:41237), 5b484d024a104088a9cb1a0f34d2a58d (127.9.33.66:34365)
I20260928 14:46:51.858623 29325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc0171a552ef47a38b936bd1b970966b" candidate_uuid: "70e2936002754f0b93399ecaacc2db2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b484d024a104088a9cb1a0f34d2a58d" is_pre_election: true
I20260928 14:46:51.858752 29325 raft_consensus.cc:2454] T fc0171a552ef47a38b936bd1b970966b P 5b484d024a104088a9cb1a0f34d2a58d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70e2936002754f0b93399ecaacc2db2e in term 0.
I20260928 14:46:51.859302 29522 raft_consensus.cc:504] T fc0171a552ef47a38b936bd1b970966b P 70e2936002754f0b93399ecaacc2db2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e2936002754f0b93399ecaacc2db2e" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41237 } } peers { permanent_uuid: "073ca38165d84135a2e04c45a80fd517" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34517 } } peers { permanent_uuid: "5b484d024a104088a9cb1a0f34d2a58d" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 34365 } }
I20260928 14:46:51.859436 29522 leader_election.cc:288] T fc0171a552ef47a38b936bd1b970966b P 70e2936002754f0b93399ecaacc2db2e [CANDIDATE]: Term 1 election: Requested vote from peers 073ca38165d84135a2e04c45a80fd517 (127.9.33.67:34517), 5b484d024a104088a9cb1a0f34d2a58d (127.9.33.66:34365)
I20260928 14:46:51.859644 29456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc0171a552ef47a38b936bd1b970966b" candidate_uuid: "70e2936002754f0b93399ecaacc2db2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "073ca38165d84135a2e04c45a80fd517"
I20260928 14:46:51.859741 29456 raft_consensus.cc:3037] T fc0171a552ef47a38b936bd1b970966b P 073ca38165d84135a2e04c45a80fd517 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:46:51.860742 29456 raft_consensus.cc:2454] T fc0171a552ef47a38b936bd1b970966b P 073ca38165d84135a2e04c45a80fd517 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70e2936002754f0b93399ecaacc2db2e in term 1.
I20260928 14:46:51.860920 29130 leader_election.cc:302] T fc0171a552ef47a38b936bd1b970966b P 70e2936002754f0b93399ecaacc2db2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 073ca38165d84135a2e04c45a80fd517, 70e2936002754f0b93399ecaacc2db2e; no voters:
I20260928 14:46:51.861048 29325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc0171a552ef47a38b936bd1b970966b" candidate_uuid: "70e2936002754f0b93399ecaacc2db2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b484d024a104088a9cb1a0f34d2a58d"
I20260928 14:46:51.861097 29522 raft_consensus.cc:2781] T fc0171a552ef47a38b936bd1b970966b P 70e2936002754f0b93399ecaacc2db2e [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:46:51.861135 29325 raft_consensus.cc:3037] T fc0171a552ef47a38b936bd1b970966b P 5b484d024a104088a9cb1a0f34d2a58d [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:46:51.861249 29522 raft_consensus.cc:686] T fc0171a552ef47a38b936bd1b970966b P 70e2936002754f0b93399ecaacc2db2e [term 1 LEADER]: Becoming Leader. State: Replica: 70e2936002754f0b93399ecaacc2db2e, State: Running, Role: LEADER
I20260928 14:46:51.861344 29522 consensus_queue.cc:237] T fc0171a552ef47a38b936bd1b970966b P 70e2936002754f0b93399ecaacc2db2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e2936002754f0b93399ecaacc2db2e" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41237 } } peers { permanent_uuid: "073ca38165d84135a2e04c45a80fd517" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34517 } } peers { permanent_uuid: "5b484d024a104088a9cb1a0f34d2a58d" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 34365 } }
I20260928 14:46:51.861835 29504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.858377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.66:34365 (local address 127.9.33.67:60351)
0928 14:46:51.858540 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:51.858552 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:51.858594 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:51.858873 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:51.858874 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:51.858876 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:46:51.858931 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:51.858933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.859831 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.859832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:51.860780 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:51.860782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.861589 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.861590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:51.861660 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:51.861665 (+ 5us) client_negotiation.cc:769] Sending connection context
0928 14:46:51.861717 (+ 52us) client_negotiation.cc:240] Negotiation successful
0928 14:46:51.861760 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142}
I20260928 14:46:51.862043 29325 raft_consensus.cc:2454] T fc0171a552ef47a38b936bd1b970966b P 5b484d024a104088a9cb1a0f34d2a58d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70e2936002754f0b93399ecaacc2db2e in term 1.
I20260928 14:46:51.862349 29512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.858577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:42133 (local address 127.9.33.65:41237)
0928 14:46:51.858615 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:51.858617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:51.859148 (+ 531us) server_negotiation.cc:407] Connection header received
0928 14:46:51.859196 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:51.859197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:51.859205 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:51.859232 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:46:51.859446 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.860171 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.861443 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.861955 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.862014 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:51.862023 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:51.862237 (+ 214us) server_negotiation.cc:299] Negotiation successful
0928 14:46:51.862283 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 14:46:51.862229 29050 catalog_manager.cc:5495] T fc0171a552ef47a38b936bd1b970966b P 70e2936002754f0b93399ecaacc2db2e reported cstate change: term changed from 0 to 1, leader changed from <none> to 70e2936002754f0b93399ecaacc2db2e (127.9.33.65). New cstate: current_term: 1 leader_uuid: "70e2936002754f0b93399ecaacc2db2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e2936002754f0b93399ecaacc2db2e" member_type: VOTER last_known_addr { host: "127.9.33.65" port: 41237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "073ca38165d84135a2e04c45a80fd517" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b484d024a104088a9cb1a0f34d2a58d" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 34365 } health_report { overall_health: UNKNOWN } } }
I20260928 14:46:51.862607 29526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.858391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:41237 (local address 127.9.33.67:42133)
0928 14:46:51.859074 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:51.859082 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0928 14:46:51.859139 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:46:51.859287 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:46:51.859287 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:46:51.859288 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0928 14:46:51.859345 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:51.859346 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.860272 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.860273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:51.861343 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:46:51.861345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.862088 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.862088 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:46:51.862159 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:51.862162 (+ 3us) client_negotiation.cc:769] Sending connection context
0928 14:46:51.862447 (+ 285us) client_negotiation.cc:240] Negotiation successful
0928 14:46:51.862507 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"thread_start_us":107,"threads_started":1}
I20260928 14:46:51.862677 29515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.858702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.33.67:60351 (local address 127.9.33.66:34365)
0928 14:46:51.858735 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:51.858737 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:51.858743 (+ 6us) server_negotiation.cc:407] Connection header received
0928 14:46:51.858761 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:51.858761 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:51.858769 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:51.858836 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0928 14:46:51.859022 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.859713 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.860897 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.861477 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.861546 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:51.861555 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:51.861976 (+ 421us) server_negotiation.cc:299] Negotiation successful
0928 14:46:51.862033 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 14:46:51.862815 29325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc0171a552ef47a38b936bd1b970966b" candidate_uuid: "073ca38165d84135a2e04c45a80fd517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b484d024a104088a9cb1a0f34d2a58d" is_pre_election: true
I20260928 14:46:51.862896 29325 raft_consensus.cc:2379] T fc0171a552ef47a38b936bd1b970966b P 5b484d024a104088a9cb1a0f34d2a58d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 073ca38165d84135a2e04c45a80fd517 in current term 1: Already voted for candidate 70e2936002754f0b93399ecaacc2db2e in this term.
I20260928 14:46:51.862843 29194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc0171a552ef47a38b936bd1b970966b" candidate_uuid: "073ca38165d84135a2e04c45a80fd517" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70e2936002754f0b93399ecaacc2db2e" is_pre_election: true
I20260928 14:46:51.863181 29392 leader_election.cc:302] T fc0171a552ef47a38b936bd1b970966b P 073ca38165d84135a2e04c45a80fd517 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 073ca38165d84135a2e04c45a80fd517; no voters: 5b484d024a104088a9cb1a0f34d2a58d, 70e2936002754f0b93399ecaacc2db2e
I20260928 14:46:51.863310 29523 raft_consensus.cc:2726] T fc0171a552ef47a38b936bd1b970966b P 073ca38165d84135a2e04c45a80fd517 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260928 14:46:51.870380 29512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.867876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50844 (local address 127.9.33.65:41237)
0928 14:46:51.867928 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:51.867930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:51.867969 (+ 39us) server_negotiation.cc:407] Connection header received
0928 14:46:51.868030 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:51.868031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:51.868047 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:51.868113 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:46:51.868219 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.868687 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.869723 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:51.869912 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:51.869979 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:51.870091 (+ 112us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:46:51.870190 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:46:51.870249 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:51.870287 (+ 38us) server_negotiation.cc:299] Negotiation successful
0928 14:46:51.870303 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260928 14:46:51.872335 29325 raft_consensus.cc:1261] T fc0171a552ef47a38b936bd1b970966b P 5b484d024a104088a9cb1a0f34d2a58d [term 1 FOLLOWER]: Refusing update from remote peer 70e2936002754f0b93399ecaacc2db2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:46:51.872371 29456 raft_consensus.cc:1261] T fc0171a552ef47a38b936bd1b970966b P 073ca38165d84135a2e04c45a80fd517 [term 1 FOLLOWER]: Refusing update from remote peer 70e2936002754f0b93399ecaacc2db2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:46:51.872671 29522 consensus_queue.cc:1035] T fc0171a552ef47a38b936bd1b970966b P 70e2936002754f0b93399ecaacc2db2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b484d024a104088a9cb1a0f34d2a58d" member_type: VOTER last_known_addr { host: "127.9.33.66" port: 34365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:46:51.872808 29527 consensus_queue.cc:1035] T fc0171a552ef47a38b936bd1b970966b P 70e2936002754f0b93399ecaacc2db2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "073ca38165d84135a2e04c45a80fd517" member_type: VOTER last_known_addr { host: "127.9.33.67" port: 34517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:46:51.875959 29532 mvcc.cc:205] Tried to move back new op lower bound from 7334325501426237440 to 7334325501384687616. Current Snapshot: MvccSnapshot[applied={T|T < 7334325501426237440}]
I20260928 14:46:51.877902 29052 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:46:51.877983 29052 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260928 14:46:51.878710 29052 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 529f7a91462d4e5a9e717249ac3dcbff: Sending DeleteTablet for 3 replicas of tablet fc0171a552ef47a38b936bd1b970966b
I20260928 14:46:51.879137 29436 tablet_service.cc:1514] Processing DeleteTablet for tablet fc0171a552ef47a38b936bd1b970966b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:46:51 UTC) from {username='slave'} at 127.0.0.1:40580
I20260928 14:46:51.879148 29305 tablet_service.cc:1514] Processing DeleteTablet for tablet fc0171a552ef47a38b936bd1b970966b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:46:51 UTC) from {username='slave'} at 127.0.0.1:32866
I20260928 14:46:51.879365 29174 tablet_service.cc:1514] Processing DeleteTablet for tablet fc0171a552ef47a38b936bd1b970966b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-28 14:46:51 UTC) from {username='slave'} at 127.0.0.1:50832
I20260928 14:46:51.879655 29541 tablet_replica.cc:330] T fc0171a552ef47a38b936bd1b970966b P 70e2936002754f0b93399ecaacc2db2e: stopping tablet replica
I20260928 14:46:51.879873 29541 raft_consensus.cc:2229] T fc0171a552ef47a38b936bd1b970966b P 70e2936002754f0b93399ecaacc2db2e [term 1 LEADER]: Raft consensus shutting down.
I20260928 14:46:51.879918 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 29112
I20260928 14:46:51.880674 29540 tablet_replica.cc:330] T fc0171a552ef47a38b936bd1b970966b P 5b484d024a104088a9cb1a0f34d2a58d: stopping tablet replica
I20260928 14:46:51.880798 29540 raft_consensus.cc:2229] T fc0171a552ef47a38b936bd1b970966b P 5b484d024a104088a9cb1a0f34d2a58d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:46:51.880919 29540 raft_consensus.cc:2258] T fc0171a552ef47a38b936bd1b970966b P 5b484d024a104088a9cb1a0f34d2a58d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:46:51.881340 29540 ts_tablet_manager.cc:1905] T fc0171a552ef47a38b936bd1b970966b P 5b484d024a104088a9cb1a0f34d2a58d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:46:51.882515 29539 tablet_replica.cc:330] T fc0171a552ef47a38b936bd1b970966b P 073ca38165d84135a2e04c45a80fd517: stopping tablet replica
I20260928 14:46:51.882651 29539 raft_consensus.cc:2229] T fc0171a552ef47a38b936bd1b970966b P 073ca38165d84135a2e04c45a80fd517 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260928 14:46:51.882805 29539 raft_consensus.cc:2258] T fc0171a552ef47a38b936bd1b970966b P 073ca38165d84135a2e04c45a80fd517 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260928 14:46:51.882880 29540 ts_tablet_manager.cc:1918] T fc0171a552ef47a38b936bd1b970966b P 5b484d024a104088a9cb1a0f34d2a58d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:46:51.882952 29540 log.cc:1196] T fc0171a552ef47a38b936bd1b970966b P 5b484d024a104088a9cb1a0f34d2a58d: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-1/wal/wals/fc0171a552ef47a38b936bd1b970966b
I20260928 14:46:51.883221 29540 ts_tablet_manager.cc:1939] T fc0171a552ef47a38b936bd1b970966b P 5b484d024a104088a9cb1a0f34d2a58d: Deleting consensus metadata
I20260928 14:46:51.883282 29539 ts_tablet_manager.cc:1905] T fc0171a552ef47a38b936bd1b970966b P 073ca38165d84135a2e04c45a80fd517: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260928 14:46:51.883694 29036 catalog_manager.cc:4841] TS 5b484d024a104088a9cb1a0f34d2a58d (127.9.33.66:34365): tablet fc0171a552ef47a38b936bd1b970966b (table test-table [id=c9cc09a34da7420fb867cb407fd94900]) successfully deleted
I20260928 14:46:51.884593 29539 ts_tablet_manager.cc:1918] T fc0171a552ef47a38b936bd1b970966b P 073ca38165d84135a2e04c45a80fd517: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260928 14:46:51.884647 29539 log.cc:1196] T fc0171a552ef47a38b936bd1b970966b P 073ca38165d84135a2e04c45a80fd517: Deleting WAL directory at /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790606561545179-9349-0/minicluster-data/ts-2/wal/wals/fc0171a552ef47a38b936bd1b970966b
I20260928 14:46:51.884884 29539 ts_tablet_manager.cc:1939] T fc0171a552ef47a38b936bd1b970966b P 073ca38165d84135a2e04c45a80fd517: Deleting consensus metadata
I20260928 14:46:51.885290 29039 catalog_manager.cc:4841] TS 073ca38165d84135a2e04c45a80fd517 (127.9.33.67:34517): tablet fc0171a552ef47a38b936bd1b970966b (table test-table [id=c9cc09a34da7420fb867cb407fd94900]) successfully deleted
W20260928 14:46:51.885993 29039 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.33.65:41237 (error 108)
I20260928 14:46:51.886196 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 29243
I20260928 14:46:51.886758 29510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:51.886343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.65:41237 (local address 127.0.0.1:50850)
0928 14:46:51.886681 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0928 14:46:51.886729 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.65:41237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318}
W20260928 14:46:51.886883 29039 catalog_manager.cc:4584] TS 70e2936002754f0b93399ecaacc2db2e (127.9.33.65:41237): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fc0171a552ef47a38b936bd1b970966b: Network error: Client connection negotiation failed: client connection to 127.9.33.65:41237: connect: Connection refused (error 111)
I20260928 14:46:51.891686 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 29374
I20260928 14:46:51.896456 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 29021
2026-09-28T14:46:51Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (719 ms)
[----------] 6 tests from EncryptionPolicyTest (8611 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:46:51 dist-test-slave-pv0w krb5kdc[29545](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:46:51 dist-test-slave-pv0w krb5kdc[29545](info): set up 2 sockets
Sep 28 14:46:51 dist-test-slave-pv0w krb5kdc[29545](info): commencing operation
krb5kdc: starting...
W20260928 14:46:52.936575 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:46:53 dist-test-slave-pv0w krb5kdc[29545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:46:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:46:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 14:46:53.063146 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40149
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:46:53.146075 29558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:46:53.146303 29558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:46:53.146328 29558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:46:53.147490 29558 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:46:53.147527 29558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:46:53.147541 29558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:46:53.147554 29558 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:46:53.147567 29558 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:46:53.149212 29558 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40149
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:46:53.149430 29558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:46:53.149622 29558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:46:53.154874 29564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:53.154874 29566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:53.154897 29563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:53.155368 29558 server_base.cc:1034] running on GCE node
I20260928 14:46:53.155660 29558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:46:53.156018 29558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:46:53.157203 29558 hybrid_clock.cc:648] HybridClock initialized: now 1790606813157192 us; error 45 us; skew 500 ppm
Sep 28 14:46:53 dist-test-slave-pv0w krb5kdc[29545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606813, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:46:53.159936 29558 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 14:46:53.160477 29558 webserver.cc:460] Webserver started at http://127.0.0.1:38123/ using document root <none> and password file <none>
I20260928 14:46:53.160753 29558 fs_manager.cc:359] Metadata directory not provided
I20260928 14:46:53.160820 29558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:46:53.160956 29558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:46:53.162168 29558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "9138b61611bf42009e9184d24738b834"
format_stamp: "Formatted at 2026-09-28 14:46:53 on dist-test-slave-pv0w"
server_key: "8af72044a396c4950649f828e06b2a41"
server_key_iv: "0bd34726db7619083dca347b3212e160"
server_key_version: "encryptionkey@0"
I20260928 14:46:53.162539 29558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "9138b61611bf42009e9184d24738b834"
format_stamp: "Formatted at 2026-09-28 14:46:53 on dist-test-slave-pv0w"
server_key: "8af72044a396c4950649f828e06b2a41"
server_key_iv: "0bd34726db7619083dca347b3212e160"
server_key_version: "encryptionkey@0"
I20260928 14:46:53.164042 29558 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:46:53.164861 29573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:53.165062 29558 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:46:53.165185 29558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "9138b61611bf42009e9184d24738b834"
format_stamp: "Formatted at 2026-09-28 14:46:53 on dist-test-slave-pv0w"
server_key: "8af72044a396c4950649f828e06b2a41"
server_key_iv: "0bd34726db7619083dca347b3212e160"
server_key_version: "encryptionkey@0"
I20260928 14:46:53.165261 29558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:46:53.181979 29558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:46:53.184095 29558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:46:53.184260 29558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:46:53.188364 29558 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40149
I20260928 14:46:53.188409 29625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40149 every 8 connection(s)
I20260928 14:46:53.188792 29558 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:46:53.189838 29626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:46:53.192306 29626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9138b61611bf42009e9184d24738b834: Bootstrap starting.
I20260928 14:46:53.192910 29626 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9138b61611bf42009e9184d24738b834: Neither blocks nor log segments found. Creating new log.
I20260928 14:46:53.193202 29626 log.cc:824] T 00000000000000000000000000000000 P 9138b61611bf42009e9184d24738b834: Log is configured to *not* fsync() on all Append() calls
I20260928 14:46:53.193908 29626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9138b61611bf42009e9184d24738b834: No bootstrap required, opened a new log
I20260928 14:46:53.195494 29626 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9138b61611bf42009e9184d24738b834 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9138b61611bf42009e9184d24738b834" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40149 } }
I20260928 14:46:53.195688 29626 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9138b61611bf42009e9184d24738b834 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:46:53.195712 29626 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9138b61611bf42009e9184d24738b834 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9138b61611bf42009e9184d24738b834, State: Initialized, Role: FOLLOWER
I20260928 14:46:53.195919 29626 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9138b61611bf42009e9184d24738b834 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9138b61611bf42009e9184d24738b834" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40149 } }
I20260928 14:46:53.196002 29626 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9138b61611bf42009e9184d24738b834 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:46:53.196027 29626 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9138b61611bf42009e9184d24738b834 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:46:53.196062 29626 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9138b61611bf42009e9184d24738b834 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:46:53.196549 29626 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9138b61611bf42009e9184d24738b834 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9138b61611bf42009e9184d24738b834" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40149 } }
I20260928 14:46:53.196633 29626 leader_election.cc:302] T 00000000000000000000000000000000 P 9138b61611bf42009e9184d24738b834 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9138b61611bf42009e9184d24738b834; no voters:
I20260928 14:46:53.196789 29626 leader_election.cc:288] T 00000000000000000000000000000000 P 9138b61611bf42009e9184d24738b834 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:46:53.196825 29629 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9138b61611bf42009e9184d24738b834 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:46:53.196945 29629 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9138b61611bf42009e9184d24738b834 [term 1 LEADER]: Becoming Leader. State: Replica: 9138b61611bf42009e9184d24738b834, State: Running, Role: LEADER
I20260928 14:46:53.197054 29626 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9138b61611bf42009e9184d24738b834 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:46:53.197034 29629 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9138b61611bf42009e9184d24738b834 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9138b61611bf42009e9184d24738b834" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40149 } }
I20260928 14:46:53.197532 29630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9138b61611bf42009e9184d24738b834 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9138b61611bf42009e9184d24738b834" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9138b61611bf42009e9184d24738b834" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40149 } } }
I20260928 14:46:53.197636 29630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9138b61611bf42009e9184d24738b834 [sys.catalog]: This master's current role is: LEADER
I20260928 14:46:53.197788 29631 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9138b61611bf42009e9184d24738b834 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9138b61611bf42009e9184d24738b834. Latest consensus state: current_term: 1 leader_uuid: "9138b61611bf42009e9184d24738b834" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9138b61611bf42009e9184d24738b834" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40149 } } }
I20260928 14:46:53.197928 29631 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9138b61611bf42009e9184d24738b834 [sys.catalog]: This master's current role is: LEADER
I20260928 14:46:53.198246 29638 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:46:53.198597 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 29558
I20260928 14:46:53.198714 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:46:53.198798 29638 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:46:53.198935 9349 external_mini_cluster.cc:1428] Setting key a0dd0a6e89bceebf2c63d202ca41006b
I20260928 14:46:53.200806 29638 catalog_manager.cc:1347] Generated new cluster ID: a56daa19f63b47cdb9de97eb0bfe5d54
I20260928 14:46:53.200868 29638 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:46:53.205567 29638 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:46:53.206300 29638 catalog_manager.cc:1504] Loading token signing keys...
Sep 28 14:46:53 dist-test-slave-pv0w krb5kdc[29545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260928 14:46:53.210788 29648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:53.199434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44695 (local address 127.0.0.1:40149)
0928 14:46:53.199651 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:53.199662 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:53.199711 (+ 49us) server_negotiation.cc:407] Connection header received
0928 14:46:53.199858 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:53.199866 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:53.199920 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:53.200068 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:53.200772 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:53.202174 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:53.203284 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:53.203921 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:53.204034 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:53.206999 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:53.207003 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:53.207003 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:46:53.207016 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:53.209059 (+ 2043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:53.209530 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:53.209533 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:53.209533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:53.209639 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:53.209898 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:53.209899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:53.209899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:53.210180 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:53.210331 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:53.210458 (+ 127us) server_negotiation.cc:299] Negotiation successful
0928 14:46:53.210560 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":77,"threads_started":1}
I20260928 14:46:53.211814 9349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:46:53.220924 29648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:53.213282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54085 (local address 127.0.0.1:40149)
0928 14:46:53.213318 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:53.213320 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:53.213446 (+ 126us) server_negotiation.cc:407] Connection header received
0928 14:46:53.213510 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:53.213510 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:53.213519 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:53.213577 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:53.214210 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:53.214991 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:53.216109 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:53.216422 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:53.216484 (+ 62us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:53.217983 (+ 1499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:53.217984 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:53.217984 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:46:53.217992 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:53.219554 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:53.220062 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:53.220063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:53.220064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:53.220124 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:53.220379 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:53.220380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:53.220380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:53.220502 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:53.220606 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:53.220759 (+ 153us) server_negotiation.cc:299] Negotiation successful
0928 14:46:53.220827 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260928 14:46:53.225396 29638 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9138b61611bf42009e9184d24738b834: Generated new TSK 0
I20260928 14:46:53.225620 29638 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:46:53.228223 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 29558
W20260928 14:46:53.232869 29543 connection.cc:535] client connection to 127.0.0.1:40149 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40149): Connection reset by peer (error 104)
2026-09-28T14:46:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1329 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:46:53 dist-test-slave-pv0w krb5kdc[29657](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:46:53 dist-test-slave-pv0w krb5kdc[29657](info): set up 2 sockets
Sep 28 14:46:53 dist-test-slave-pv0w krb5kdc[29657](info): commencing operation
krb5kdc: starting...
W20260928 14:46:54.276649 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:46:54 dist-test-slave-pv0w krb5kdc[29657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606814, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:46:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:46:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 14:46:54.399605 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39211
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39211
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:46:54.483165 29670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:46:54.483400 29670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:46:54.483426 29670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:46:54.484608 29670 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:46:54.484647 29670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:46:54.484663 29670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:46:54.484675 29670 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:46:54.484687 29670 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:46:54.486125 29670 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39211
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29670
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:46:54.486334 29670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:46:54.486524 29670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:46:54.491850 29678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:54.491858 29675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:54.491886 29676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:54.491890 29670 server_base.cc:1034] running on GCE node
I20260928 14:46:54.492412 29670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:46:54.492720 29670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:46:54.493891 29670 hybrid_clock.cc:648] HybridClock initialized: now 1790606814493877 us; error 42 us; skew 500 ppm
Sep 28 14:46:54 dist-test-slave-pv0w krb5kdc[29657](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606814, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:46:54.503661 29670 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 14:46:54.504176 29670 webserver.cc:460] Webserver started at http://127.0.0.1:35549/ using document root <none> and password file <none>
I20260928 14:46:54.504421 29670 fs_manager.cc:359] Metadata directory not provided
I20260928 14:46:54.504489 29670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:46:54.504644 29670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:46:54.505781 29670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "f1eb6acc03744665809721db76280ded"
format_stamp: "Formatted at 2026-09-28 14:46:54 on dist-test-slave-pv0w"
server_key: "24cfbaa9310d3a1c868fbbb2742346cc"
server_key_iv: "edfa8799a2d46928bbc1713554040105"
server_key_version: "encryptionkey@0"
I20260928 14:46:54.506157 29670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "f1eb6acc03744665809721db76280ded"
format_stamp: "Formatted at 2026-09-28 14:46:54 on dist-test-slave-pv0w"
server_key: "24cfbaa9310d3a1c868fbbb2742346cc"
server_key_iv: "edfa8799a2d46928bbc1713554040105"
server_key_version: "encryptionkey@0"
I20260928 14:46:54.507891 29670 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260928 14:46:54.508754 29685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:54.508955 29670 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:46:54.509057 29670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "f1eb6acc03744665809721db76280ded"
format_stamp: "Formatted at 2026-09-28 14:46:54 on dist-test-slave-pv0w"
server_key: "24cfbaa9310d3a1c868fbbb2742346cc"
server_key_iv: "edfa8799a2d46928bbc1713554040105"
server_key_version: "encryptionkey@0"
I20260928 14:46:54.509145 29670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:46:54.527510 29670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:46:54.529992 29670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:46:54.530169 29670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:46:54.534355 29670 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39211
I20260928 14:46:54.534392 29737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39211 every 8 connection(s)
I20260928 14:46:54.534787 29670 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:46:54.535012 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 29670
I20260928 14:46:54.535161 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:46:54.535393 9349 external_mini_cluster.cc:1428] Setting key 0ee590831b271036aca591985e096ce6
I20260928 14:46:54.536178 29738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:46:54.539376 29738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1eb6acc03744665809721db76280ded: Bootstrap starting.
I20260928 14:46:54.540136 29738 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1eb6acc03744665809721db76280ded: Neither blocks nor log segments found. Creating new log.
I20260928 14:46:54.540480 29738 log.cc:824] T 00000000000000000000000000000000 P f1eb6acc03744665809721db76280ded: Log is configured to *not* fsync() on all Append() calls
I20260928 14:46:54.541314 29738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1eb6acc03744665809721db76280ded: No bootstrap required, opened a new log
I20260928 14:46:54.541519 29741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:54.535986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34137 (local address 127.0.0.1:39211)
0928 14:46:54.536184 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:54.536194 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:54.536227 (+ 33us) server_negotiation.cc:407] Connection header received
0928 14:46:54.536715 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:54.536721 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:54.536769 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:54.536889 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:54.537168 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:54.538071 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:54.539198 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:54.539830 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:54.539919 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:54.540766 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:54.540771 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:54.540771 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:46:54.540943 (+ 172us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:54.540981 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:46:54.540992 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:54.541058 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:54.541180 (+ 122us) server_negotiation.cc:299] Negotiation successful
0928 14:46:54.541222 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":91,"threads_started":1}
I20260928 14:46:54.543128 9349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:46:54.543870 29738 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1eb6acc03744665809721db76280ded [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1eb6acc03744665809721db76280ded" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39211 } }
I20260928 14:46:54.544020 29738 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1eb6acc03744665809721db76280ded [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:46:54.544044 29738 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1eb6acc03744665809721db76280ded [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1eb6acc03744665809721db76280ded, State: Initialized, Role: FOLLOWER
I20260928 14:46:54.544186 29738 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1eb6acc03744665809721db76280ded [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1eb6acc03744665809721db76280ded" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39211 } }
I20260928 14:46:54.544277 29738 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1eb6acc03744665809721db76280ded [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:46:54.544308 29738 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1eb6acc03744665809721db76280ded [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:46:54.544361 29738 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1eb6acc03744665809721db76280ded [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:46:54.545065 29738 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1eb6acc03744665809721db76280ded [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1eb6acc03744665809721db76280ded" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39211 } }
I20260928 14:46:54.545189 29738 leader_election.cc:302] T 00000000000000000000000000000000 P f1eb6acc03744665809721db76280ded [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1eb6acc03744665809721db76280ded; no voters:
I20260928 14:46:54.545394 29738 leader_election.cc:288] T 00000000000000000000000000000000 P f1eb6acc03744665809721db76280ded [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:46:54.545426 29748 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1eb6acc03744665809721db76280ded [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:46:54.545542 29748 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1eb6acc03744665809721db76280ded [term 1 LEADER]: Becoming Leader. State: Replica: f1eb6acc03744665809721db76280ded, State: Running, Role: LEADER
I20260928 14:46:54.545629 29748 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1eb6acc03744665809721db76280ded [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1eb6acc03744665809721db76280ded" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39211 } }
I20260928 14:46:54.545689 29738 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1eb6acc03744665809721db76280ded [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:46:54.545960 29748 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1eb6acc03744665809721db76280ded [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f1eb6acc03744665809721db76280ded" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1eb6acc03744665809721db76280ded" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39211 } } }
I20260928 14:46:54.546046 29748 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1eb6acc03744665809721db76280ded [sys.catalog]: This master's current role is: LEADER
I20260928 14:46:54.546229 29748 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1eb6acc03744665809721db76280ded [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1eb6acc03744665809721db76280ded. Latest consensus state: current_term: 1 leader_uuid: "f1eb6acc03744665809721db76280ded" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1eb6acc03744665809721db76280ded" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39211 } } }
I20260928 14:46:54.546308 29748 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1eb6acc03744665809721db76280ded [sys.catalog]: This master's current role is: LEADER
I20260928 14:46:54.546273 29752 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:46:54.546891 29752 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:46:54.548619 29752 catalog_manager.cc:1347] Generated new cluster ID: 31abb3e835db42c09a83785feb2666a7
I20260928 14:46:54.548693 29752 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:46:54.548799 29741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:54.544649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51635 (local address 127.0.0.1:39211)
0928 14:46:54.544686 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:54.544688 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:54.544811 (+ 123us) server_negotiation.cc:407] Connection header received
0928 14:46:54.544887 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:54.544888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:54.544900 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:54.544943 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:54.545092 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:54.545849 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:54.546910 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:54.547175 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:54.547222 (+ 47us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:54.548310 (+ 1088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:54.548311 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:54.548311 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:46:54.548316 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:54.548332 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:46:54.548337 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:54.548368 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:54.548575 (+ 207us) server_negotiation.cc:299] Negotiation successful
0928 14:46:54.548590 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260928 14:46:54.556494 29752 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:46:54.557067 29752 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:46:54.561232 29752 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1eb6acc03744665809721db76280ded: Generated new TSK 0
I20260928 14:46:54.561439 29752 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:46:54.566236 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 29670
W20260928 14:46:54.570763 29655 connection.cc:535] client connection to 127.0.0.1:39211 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39211): Connection reset by peer (error 104)
2026-09-28T14:46:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1337 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:46:54 dist-test-slave-pv0w krb5kdc[29769](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:46:54 dist-test-slave-pv0w krb5kdc[29769](info): set up 2 sockets
Sep 28 14:46:54 dist-test-slave-pv0w krb5kdc[29769](info): commencing operation
krb5kdc: starting...
W20260928 14:46:55.672662 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:46:55 dist-test-slave-pv0w krb5kdc[29769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:46:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:46:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 14:46:55.800280 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46855
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46855
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:46:55.885641 29782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:46:55.885862 29782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:46:55.885888 29782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:46:55.887130 29782 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:46:55.887176 29782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:46:55.887192 29782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:46:55.887204 29782 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:46:55.887216 29782 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:46:55.888896 29782 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46855
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:46:55.889175 29782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:46:55.889418 29782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:46:55.894582 29790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:55.894645 29787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:55.894645 29788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:55.894802 29782 server_base.cc:1034] running on GCE node
I20260928 14:46:55.895191 29782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:46:55.895539 29782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:46:55.896727 29782 hybrid_clock.cc:648] HybridClock initialized: now 1790606815896712 us; error 32 us; skew 500 ppm
Sep 28 14:46:55 dist-test-slave-pv0w krb5kdc[29769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606815, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:46:55.899585 29782 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 14:46:55.900122 29782 webserver.cc:460] Webserver started at http://127.0.0.1:38199/ using document root <none> and password file <none>
I20260928 14:46:55.900344 29782 fs_manager.cc:359] Metadata directory not provided
I20260928 14:46:55.900388 29782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:46:55.900484 29782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:46:55.901546 29782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "f5204ce67c2c4669b86c9c9a13ef59e9"
format_stamp: "Formatted at 2026-09-28 14:46:55 on dist-test-slave-pv0w"
server_key: "89f0a5ce43c87d5757de112ffb65ec98"
server_key_iv: "5b8b6b91212119dc11adf2e3adc278b0"
server_key_version: "encryptionkey@0"
I20260928 14:46:55.901850 29782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "f5204ce67c2c4669b86c9c9a13ef59e9"
format_stamp: "Formatted at 2026-09-28 14:46:55 on dist-test-slave-pv0w"
server_key: "89f0a5ce43c87d5757de112ffb65ec98"
server_key_iv: "5b8b6b91212119dc11adf2e3adc278b0"
server_key_version: "encryptionkey@0"
I20260928 14:46:55.903178 29782 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:46:55.903998 29797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:55.904239 29782 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:46:55.904357 29782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "f5204ce67c2c4669b86c9c9a13ef59e9"
format_stamp: "Formatted at 2026-09-28 14:46:55 on dist-test-slave-pv0w"
server_key: "89f0a5ce43c87d5757de112ffb65ec98"
server_key_iv: "5b8b6b91212119dc11adf2e3adc278b0"
server_key_version: "encryptionkey@0"
I20260928 14:46:55.904436 29782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:46:55.907960 29782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:46:55.910085 29782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:46:55.910254 29782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:46:55.914425 29782 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46855
I20260928 14:46:55.914458 29849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46855 every 8 connection(s)
I20260928 14:46:55.914875 29782 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:46:55.915527 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 29782
I20260928 14:46:55.915668 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:46:55.915908 9349 external_mini_cluster.cc:1428] Setting key a3da8fe469e2577d7df43b05d14fc6b2
I20260928 14:46:55.916031 29850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:46:55.918504 29853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:55.916422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38315 (local address 127.0.0.1:46855)
0928 14:46:55.916589 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:55.916598 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:55.916631 (+ 33us) server_negotiation.cc:407] Connection header received
0928 14:46:55.917164 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:55.917173 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:55.917208 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:55.917372 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:55.918068 (+ 696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:55.918072 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:55.918072 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:46:55.918076 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:55.918122 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:46:55.918136 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:55.918214 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:55.918272 (+ 58us) server_negotiation.cc:299] Negotiation successful
0928 14:46:55.918315 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":76,"threads_started":1}
I20260928 14:46:55.919796 9349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:46:55.919833 29850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5204ce67c2c4669b86c9c9a13ef59e9: Bootstrap starting.
I20260928 14:46:55.920576 29850 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5204ce67c2c4669b86c9c9a13ef59e9: Neither blocks nor log segments found. Creating new log.
I20260928 14:46:55.920926 29850 log.cc:824] T 00000000000000000000000000000000 P f5204ce67c2c4669b86c9c9a13ef59e9: Log is configured to *not* fsync() on all Append() calls
I20260928 14:46:55.921753 29850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5204ce67c2c4669b86c9c9a13ef59e9: No bootstrap required, opened a new log
I20260928 14:46:55.922515 29853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:55.921455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53481 (local address 127.0.0.1:46855)
0928 14:46:55.921495 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:55.921498 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:55.921645 (+ 147us) server_negotiation.cc:407] Connection header received
0928 14:46:55.921721 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:55.921722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:55.921726 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:55.921771 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:55.922299 (+ 528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:55.922300 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:55.922301 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:46:55.922304 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:55.922317 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:46:55.922321 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:55.922360 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:55.922415 (+ 55us) server_negotiation.cc:299] Negotiation successful
0928 14:46:55.922425 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 14:46:55.924094 29850 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5204ce67c2c4669b86c9c9a13ef59e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5204ce67c2c4669b86c9c9a13ef59e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46855 } }
I20260928 14:46:55.924242 29850 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5204ce67c2c4669b86c9c9a13ef59e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:46:55.924266 29850 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5204ce67c2c4669b86c9c9a13ef59e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5204ce67c2c4669b86c9c9a13ef59e9, State: Initialized, Role: FOLLOWER
I20260928 14:46:55.924425 29850 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5204ce67c2c4669b86c9c9a13ef59e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5204ce67c2c4669b86c9c9a13ef59e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46855 } }
I20260928 14:46:55.924521 29850 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5204ce67c2c4669b86c9c9a13ef59e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:46:55.924575 29850 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5204ce67c2c4669b86c9c9a13ef59e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:46:55.924629 29850 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5204ce67c2c4669b86c9c9a13ef59e9 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:46:55.925217 29850 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5204ce67c2c4669b86c9c9a13ef59e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5204ce67c2c4669b86c9c9a13ef59e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46855 } }
I20260928 14:46:55.925309 29850 leader_election.cc:302] T 00000000000000000000000000000000 P f5204ce67c2c4669b86c9c9a13ef59e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5204ce67c2c4669b86c9c9a13ef59e9; no voters:
I20260928 14:46:55.925520 29850 leader_election.cc:288] T 00000000000000000000000000000000 P f5204ce67c2c4669b86c9c9a13ef59e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:46:55.925608 29860 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5204ce67c2c4669b86c9c9a13ef59e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:46:55.925779 29860 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5204ce67c2c4669b86c9c9a13ef59e9 [term 1 LEADER]: Becoming Leader. State: Replica: f5204ce67c2c4669b86c9c9a13ef59e9, State: Running, Role: LEADER
I20260928 14:46:55.925817 29850 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5204ce67c2c4669b86c9c9a13ef59e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:46:55.925871 29860 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5204ce67c2c4669b86c9c9a13ef59e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5204ce67c2c4669b86c9c9a13ef59e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46855 } }
I20260928 14:46:55.926344 29861 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5204ce67c2c4669b86c9c9a13ef59e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5204ce67c2c4669b86c9c9a13ef59e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5204ce67c2c4669b86c9c9a13ef59e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46855 } } }
I20260928 14:46:55.926360 29862 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5204ce67c2c4669b86c9c9a13ef59e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5204ce67c2c4669b86c9c9a13ef59e9. Latest consensus state: current_term: 1 leader_uuid: "f5204ce67c2c4669b86c9c9a13ef59e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5204ce67c2c4669b86c9c9a13ef59e9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46855 } } }
I20260928 14:46:55.926460 29861 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5204ce67c2c4669b86c9c9a13ef59e9 [sys.catalog]: This master's current role is: LEADER
I20260928 14:46:55.926466 29862 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5204ce67c2c4669b86c9c9a13ef59e9 [sys.catalog]: This master's current role is: LEADER
I20260928 14:46:55.926667 29869 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:46:55.927366 29869 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:46:55.929129 29869 catalog_manager.cc:1347] Generated new cluster ID: 131baf954bc4459785404d058d70a63c
I20260928 14:46:55.929200 29869 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:46:55.935887 29869 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:46:55.936101 29869 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:46:55.944415 29869 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5204ce67c2c4669b86c9c9a13ef59e9: Generated new TSK 0
I20260928 14:46:55.944648 29869 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:46:55.949080 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 29782
2026-09-28T14:46:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1382 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:46:56 dist-test-slave-pv0w krb5kdc[29881](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:46:56 dist-test-slave-pv0w krb5kdc[29881](info): set up 2 sockets
Sep 28 14:46:56 dist-test-slave-pv0w krb5kdc[29881](info): commencing operation
krb5kdc: starting...
W20260928 14:46:56.992530 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:46:57 dist-test-slave-pv0w krb5kdc[29881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:46:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:46:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 14:46:57.114795 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43293
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43293
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:46:57.201715 29894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:46:57.201996 29894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:46:57.202041 29894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:46:57.203357 29894 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:46:57.203433 29894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:46:57.203459 29894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:46:57.203481 29894 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:46:57.203516 29894 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:46:57.205185 29894 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43293
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:46:57.205502 29894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:46:57.205747 29894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:46:57.211606 29899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:57.211586 29900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:57.211586 29902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:57.211622 29894 server_base.cc:1034] running on GCE node
I20260928 14:46:57.212181 29894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:46:57.212471 29894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:46:57.213683 29894 hybrid_clock.cc:648] HybridClock initialized: now 1790606817213656 us; error 32 us; skew 500 ppm
Sep 28 14:46:57 dist-test-slave-pv0w krb5kdc[29881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606817, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:46:57.216468 29894 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 14:46:57.216995 29894 webserver.cc:460] Webserver started at http://127.0.0.1:40233/ using document root <none> and password file <none>
I20260928 14:46:57.217272 29894 fs_manager.cc:359] Metadata directory not provided
I20260928 14:46:57.217320 29894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:46:57.217454 29894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:46:57.218596 29894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "ec5ba8c869ea4592ad9a089162d1b2a9"
format_stamp: "Formatted at 2026-09-28 14:46:57 on dist-test-slave-pv0w"
server_key: "af40c9dc96b2786bcea7a460416f5919"
server_key_iv: "92db65e6135d4276bd75129189b1ffca"
server_key_version: "encryptionkey@0"
I20260928 14:46:57.218979 29894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "ec5ba8c869ea4592ad9a089162d1b2a9"
format_stamp: "Formatted at 2026-09-28 14:46:57 on dist-test-slave-pv0w"
server_key: "af40c9dc96b2786bcea7a460416f5919"
server_key_iv: "92db65e6135d4276bd75129189b1ffca"
server_key_version: "encryptionkey@0"
I20260928 14:46:57.220563 29894 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:46:57.221530 29909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:57.221776 29894 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:46:57.221880 29894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "ec5ba8c869ea4592ad9a089162d1b2a9"
format_stamp: "Formatted at 2026-09-28 14:46:57 on dist-test-slave-pv0w"
server_key: "af40c9dc96b2786bcea7a460416f5919"
server_key_iv: "92db65e6135d4276bd75129189b1ffca"
server_key_version: "encryptionkey@0"
I20260928 14:46:57.221966 29894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:46:57.241909 29894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:46:57.244568 29894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:46:57.244781 29894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:46:57.249030 29894 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43293
I20260928 14:46:57.249162 29961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43293 every 8 connection(s)
I20260928 14:46:57.249492 29894 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:46:57.250622 29962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:46:57.250705 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 29894
I20260928 14:46:57.250808 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:46:57.251051 9349 external_mini_cluster.cc:1428] Setting key 856ae3f6bc985241e48d8e4a6b457333
I20260928 14:46:57.253886 29962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec5ba8c869ea4592ad9a089162d1b2a9: Bootstrap starting.
I20260928 14:46:57.254532 29962 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec5ba8c869ea4592ad9a089162d1b2a9: Neither blocks nor log segments found. Creating new log.
I20260928 14:46:57.254818 29962 log.cc:824] T 00000000000000000000000000000000 P ec5ba8c869ea4592ad9a089162d1b2a9: Log is configured to *not* fsync() on all Append() calls
I20260928 14:46:57.255555 29962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec5ba8c869ea4592ad9a089162d1b2a9: No bootstrap required, opened a new log
Sep 28 14:46:57 dist-test-slave-pv0w krb5kdc[29881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606817, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260928 14:46:57.257759 29962 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec5ba8c869ea4592ad9a089162d1b2a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec5ba8c869ea4592ad9a089162d1b2a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43293 } }
I20260928 14:46:57.257966 29962 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec5ba8c869ea4592ad9a089162d1b2a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:46:57.257997 29962 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec5ba8c869ea4592ad9a089162d1b2a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec5ba8c869ea4592ad9a089162d1b2a9, State: Initialized, Role: FOLLOWER
I20260928 14:46:57.258155 29962 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec5ba8c869ea4592ad9a089162d1b2a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec5ba8c869ea4592ad9a089162d1b2a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43293 } }
I20260928 14:46:57.258234 29962 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec5ba8c869ea4592ad9a089162d1b2a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:46:57.258257 29962 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec5ba8c869ea4592ad9a089162d1b2a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:46:57.258323 29962 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec5ba8c869ea4592ad9a089162d1b2a9 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:46:57.258917 29962 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec5ba8c869ea4592ad9a089162d1b2a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec5ba8c869ea4592ad9a089162d1b2a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43293 } }
I20260928 14:46:57.259012 29962 leader_election.cc:302] T 00000000000000000000000000000000 P ec5ba8c869ea4592ad9a089162d1b2a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec5ba8c869ea4592ad9a089162d1b2a9; no voters:
I20260928 14:46:57.259235 29962 leader_election.cc:288] T 00000000000000000000000000000000 P ec5ba8c869ea4592ad9a089162d1b2a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:46:57.259272 29967 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec5ba8c869ea4592ad9a089162d1b2a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:46:57.259378 29967 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec5ba8c869ea4592ad9a089162d1b2a9 [term 1 LEADER]: Becoming Leader. State: Replica: ec5ba8c869ea4592ad9a089162d1b2a9, State: Running, Role: LEADER
I20260928 14:46:57.259496 29967 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec5ba8c869ea4592ad9a089162d1b2a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec5ba8c869ea4592ad9a089162d1b2a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43293 } }
I20260928 14:46:57.259532 29962 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec5ba8c869ea4592ad9a089162d1b2a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:46:57.259863 29967 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec5ba8c869ea4592ad9a089162d1b2a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec5ba8c869ea4592ad9a089162d1b2a9. Latest consensus state: current_term: 1 leader_uuid: "ec5ba8c869ea4592ad9a089162d1b2a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec5ba8c869ea4592ad9a089162d1b2a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43293 } } }
I20260928 14:46:57.259871 29968 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec5ba8c869ea4592ad9a089162d1b2a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec5ba8c869ea4592ad9a089162d1b2a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec5ba8c869ea4592ad9a089162d1b2a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43293 } } }
I20260928 14:46:57.260008 29967 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec5ba8c869ea4592ad9a089162d1b2a9 [sys.catalog]: This master's current role is: LEADER
I20260928 14:46:57.260021 29968 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec5ba8c869ea4592ad9a089162d1b2a9 [sys.catalog]: This master's current role is: LEADER
I20260928 14:46:57.260305 29974 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:46:57.261480 29974 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:46:57.262890 29965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:57.251629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57601 (local address 127.0.0.1:43293)
0928 14:46:57.251838 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:57.251848 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:57.251886 (+ 38us) server_negotiation.cc:407] Connection header received
0928 14:46:57.252353 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:57.252360 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:57.252409 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:57.252566 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:57.253281 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:57.254221 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:57.255063 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:57.255494 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:57.255611 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:57.258417 (+ 2806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:57.258423 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:57.258423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:46:57.258437 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:57.260815 (+ 2378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:57.261304 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:57.261308 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:57.261309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:57.261404 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:57.261626 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:57.261628 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:57.261628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:57.261945 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:57.262104 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:57.262539 (+ 435us) server_negotiation.cc:299] Negotiation successful
0928 14:46:57.262684 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":95,"threads_started":1}
I20260928 14:46:57.263394 29974 catalog_manager.cc:1347] Generated new cluster ID: 836263f5d1a247d085582ac85fb86b24
I20260928 14:46:57.263496 29974 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:46:57.264016 9349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:46:57.270125 29974 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:46:57.270795 29974 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:46:57.274667 29965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:57.267473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55783 (local address 127.0.0.1:43293)
0928 14:46:57.267514 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:57.267516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:57.267762 (+ 246us) server_negotiation.cc:407] Connection header received
0928 14:46:57.267832 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:57.267833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:57.267843 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:57.267893 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:57.268585 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:57.269074 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:57.269851 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:57.270185 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:57.270259 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:57.271875 (+ 1616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:57.271877 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:57.271878 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:46:57.271885 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:57.273472 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:57.273878 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:57.273879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:57.273879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:57.273934 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:46:57.274134 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:46:57.274134 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:46:57.274135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:46:57.274254 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:57.274344 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:57.274482 (+ 138us) server_negotiation.cc:299] Negotiation successful
0928 14:46:57.274571 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260928 14:46:57.276448 29974 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec5ba8c869ea4592ad9a089162d1b2a9: Generated new TSK 0
I20260928 14:46:57.276639 29974 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:46:57.279376 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 29894
2026-09-28T14:46:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1330 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:46:57 dist-test-slave-pv0w krb5kdc[29993](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:46:57 dist-test-slave-pv0w krb5kdc[29993](info): set up 2 sockets
Sep 28 14:46:57 dist-test-slave-pv0w krb5kdc[29993](info): commencing operation
krb5kdc: starting...
W20260928 14:46:58.328504 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:46:58 dist-test-slave-pv0w krb5kdc[29993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606818, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:46:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:46:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 14:46:58.454934 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35021
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35021
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:46:58.538399 30006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:46:58.538631 30006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:46:58.538657 30006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:46:58.539835 30006 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:46:58.539880 30006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:46:58.539893 30006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:46:58.539907 30006 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:46:58.539919 30006 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:46:58.541388 30006 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35021
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:46:58.541606 30006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:46:58.541785 30006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:46:58.546895 30012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:58.546901 30011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:58.547117 30006 server_base.cc:1034] running on GCE node
W20260928 14:46:58.546901 30014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:58.547487 30006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:46:58.547785 30006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:46:58.548970 30006 hybrid_clock.cc:648] HybridClock initialized: now 1790606818548931 us; error 38 us; skew 500 ppm
Sep 28 14:46:58 dist-test-slave-pv0w krb5kdc[29993](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606818, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:46:58.551649 30006 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 14:46:58.552184 30006 webserver.cc:460] Webserver started at http://127.0.0.1:38975/ using document root <none> and password file <none>
I20260928 14:46:58.552423 30006 fs_manager.cc:359] Metadata directory not provided
I20260928 14:46:58.552492 30006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:46:58.552649 30006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:46:58.553779 30006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "66ff26e6be5d49d3b41e6fde10f8114e"
format_stamp: "Formatted at 2026-09-28 14:46:58 on dist-test-slave-pv0w"
server_key: "0ca5fbe01ef799b71fd4e8c5b41d7a83"
server_key_iv: "fea84a8d879cd7005816ecf0c877b10e"
server_key_version: "encryptionkey@0"
I20260928 14:46:58.554162 30006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "66ff26e6be5d49d3b41e6fde10f8114e"
format_stamp: "Formatted at 2026-09-28 14:46:58 on dist-test-slave-pv0w"
server_key: "0ca5fbe01ef799b71fd4e8c5b41d7a83"
server_key_iv: "fea84a8d879cd7005816ecf0c877b10e"
server_key_version: "encryptionkey@0"
I20260928 14:46:58.555553 30006 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:46:58.556474 30021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:58.556663 30006 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260928 14:46:58.556763 30006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "66ff26e6be5d49d3b41e6fde10f8114e"
format_stamp: "Formatted at 2026-09-28 14:46:58 on dist-test-slave-pv0w"
server_key: "0ca5fbe01ef799b71fd4e8c5b41d7a83"
server_key_iv: "fea84a8d879cd7005816ecf0c877b10e"
server_key_version: "encryptionkey@0"
I20260928 14:46:58.556846 30006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:46:58.569864 30006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:46:58.571707 30006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:46:58.571970 30006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:46:58.576030 30006 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35021
I20260928 14:46:58.576061 30073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35021 every 8 connection(s)
I20260928 14:46:58.576419 30006 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:46:58.577452 30074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:46:58.579813 30074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66ff26e6be5d49d3b41e6fde10f8114e: Bootstrap starting.
I20260928 14:46:58.580045 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 30006
I20260928 14:46:58.580178 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:46:58.580396 9349 external_mini_cluster.cc:1428] Setting key 268fd1ca34ddb39d35fec2ef9e3750a9
I20260928 14:46:58.580535 30074 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66ff26e6be5d49d3b41e6fde10f8114e: Neither blocks nor log segments found. Creating new log.
I20260928 14:46:58.580865 30074 log.cc:824] T 00000000000000000000000000000000 P 66ff26e6be5d49d3b41e6fde10f8114e: Log is configured to *not* fsync() on all Append() calls
I20260928 14:46:58.581751 30074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66ff26e6be5d49d3b41e6fde10f8114e: No bootstrap required, opened a new log
I20260928 14:46:58.584368 30074 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66ff26e6be5d49d3b41e6fde10f8114e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66ff26e6be5d49d3b41e6fde10f8114e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35021 } }
I20260928 14:46:58.584607 30074 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66ff26e6be5d49d3b41e6fde10f8114e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:46:58.584667 30074 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66ff26e6be5d49d3b41e6fde10f8114e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ff26e6be5d49d3b41e6fde10f8114e, State: Initialized, Role: FOLLOWER
I20260928 14:46:58.584824 30074 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66ff26e6be5d49d3b41e6fde10f8114e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66ff26e6be5d49d3b41e6fde10f8114e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35021 } }
I20260928 14:46:58.584913 30074 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66ff26e6be5d49d3b41e6fde10f8114e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:46:58.584956 30074 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66ff26e6be5d49d3b41e6fde10f8114e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:46:58.585022 30074 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66ff26e6be5d49d3b41e6fde10f8114e [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:46:58.585871 30074 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66ff26e6be5d49d3b41e6fde10f8114e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66ff26e6be5d49d3b41e6fde10f8114e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35021 } }
I20260928 14:46:58.586015 30074 leader_election.cc:302] T 00000000000000000000000000000000 P 66ff26e6be5d49d3b41e6fde10f8114e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66ff26e6be5d49d3b41e6fde10f8114e; no voters:
I20260928 14:46:58.586239 30074 leader_election.cc:288] T 00000000000000000000000000000000 P 66ff26e6be5d49d3b41e6fde10f8114e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:46:58.586265 30079 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66ff26e6be5d49d3b41e6fde10f8114e [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:46:58.586392 30079 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66ff26e6be5d49d3b41e6fde10f8114e [term 1 LEADER]: Becoming Leader. State: Replica: 66ff26e6be5d49d3b41e6fde10f8114e, State: Running, Role: LEADER
I20260928 14:46:58.586547 30074 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66ff26e6be5d49d3b41e6fde10f8114e [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:46:58.586566 30079 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66ff26e6be5d49d3b41e6fde10f8114e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66ff26e6be5d49d3b41e6fde10f8114e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35021 } }
I20260928 14:46:58.587051 30077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:58.580947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45119 (local address 127.0.0.1:35021)
0928 14:46:58.581150 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:58.581161 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:58.581197 (+ 36us) server_negotiation.cc:407] Connection header received
0928 14:46:58.581737 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:58.581745 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:58.581833 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:58.581979 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:58.582214 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:58.583086 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:58.584557 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:58.585196 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:58.585287 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:58.586067 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:58.586072 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:58.586072 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:46:58.586301 (+ 229us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:58.586350 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:46:58.586366 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:58.586698 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:58.586730 (+ 32us) server_negotiation.cc:299] Negotiation successful
0928 14:46:58.586803 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":88,"threads_started":1}
I20260928 14:46:58.587074 30080 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66ff26e6be5d49d3b41e6fde10f8114e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66ff26e6be5d49d3b41e6fde10f8114e. Latest consensus state: current_term: 1 leader_uuid: "66ff26e6be5d49d3b41e6fde10f8114e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66ff26e6be5d49d3b41e6fde10f8114e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35021 } } }
I20260928 14:46:58.587325 30080 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66ff26e6be5d49d3b41e6fde10f8114e [sys.catalog]: This master's current role is: LEADER
I20260928 14:46:58.587186 30081 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66ff26e6be5d49d3b41e6fde10f8114e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66ff26e6be5d49d3b41e6fde10f8114e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66ff26e6be5d49d3b41e6fde10f8114e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35021 } } }
I20260928 14:46:58.587743 30081 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66ff26e6be5d49d3b41e6fde10f8114e [sys.catalog]: This master's current role is: LEADER
I20260928 14:46:58.587966 30088 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:46:58.588215 9349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:46:58.588624 30088 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:46:58.590337 30088 catalog_manager.cc:1347] Generated new cluster ID: 3e67b6cb79c8437b8747d4300e5fb478
I20260928 14:46:58.590386 30088 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:46:58.593262 30077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:58.589788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59893 (local address 127.0.0.1:35021)
0928 14:46:58.589860 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:58.589862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:58.590117 (+ 255us) server_negotiation.cc:407] Connection header received
0928 14:46:58.590173 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:58.590173 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:58.590186 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:58.590251 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:58.590465 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:58.591019 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:58.591881 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:46:58.592122 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:46:58.592216 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:46:58.592952 (+ 736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:58.592954 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:58.592954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:46:58.592959 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:58.592974 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:46:58.592979 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:58.593053 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:58.593132 (+ 79us) server_negotiation.cc:299] Negotiation successful
0928 14:46:58.593156 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260928 14:46:58.607993 30088 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:46:58.608569 30088 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:46:58.616899 30088 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66ff26e6be5d49d3b41e6fde10f8114e: Generated new TSK 0
I20260928 14:46:58.617116 30088 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:46:58.621095 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 30006
2026-09-28T14:46:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1341 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:46:58 dist-test-slave-pv0w krb5kdc[30105](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:46:58 dist-test-slave-pv0w krb5kdc[30105](info): set up 2 sockets
Sep 28 14:46:58 dist-test-slave-pv0w krb5kdc[30105](info): commencing operation
krb5kdc: starting...
W20260928 14:46:59.656705 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:46:59 dist-test-slave-pv0w krb5kdc[30105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606819, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:46:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:46:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 14:46:59.778945 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33241
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33241
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:46:59.863456 30118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:46:59.863780 30118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:46:59.863834 30118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:46:59.865096 30118 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:46:59.865173 30118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:46:59.865199 30118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:46:59.865228 30118 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:46:59.865257 30118 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:46:59.866875 30118 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:46:59.867182 30118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:46:59.867420 30118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:46:59.872605 30126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:59.872606 30123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:46:59.872679 30124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:46:59.872871 30118 server_base.cc:1034] running on GCE node
I20260928 14:46:59.873128 30118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:46:59.873397 30118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:46:59.874563 30118 hybrid_clock.cc:648] HybridClock initialized: now 1790606819874545 us; error 35 us; skew 500 ppm
Sep 28 14:46:59 dist-test-slave-pv0w krb5kdc[30105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606819, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:46:59.884102 30118 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 14:46:59.884626 30118 webserver.cc:460] Webserver started at http://127.0.0.1:40197/ using document root <none> and password file <none>
I20260928 14:46:59.884887 30118 fs_manager.cc:359] Metadata directory not provided
I20260928 14:46:59.884956 30118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:46:59.885111 30118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:46:59.886272 30118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "056309a49057467982f9a1c646f19eff"
format_stamp: "Formatted at 2026-09-28 14:46:59 on dist-test-slave-pv0w"
server_key: "b60d8e174b7bacde24146d3fa1c8bd85"
server_key_iv: "3b7ca718e4bf96a1c12cd0ef4d1897f6"
server_key_version: "encryptionkey@0"
I20260928 14:46:59.886646 30118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "056309a49057467982f9a1c646f19eff"
format_stamp: "Formatted at 2026-09-28 14:46:59 on dist-test-slave-pv0w"
server_key: "b60d8e174b7bacde24146d3fa1c8bd85"
server_key_iv: "3b7ca718e4bf96a1c12cd0ef4d1897f6"
server_key_version: "encryptionkey@0"
I20260928 14:46:59.888087 30118 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:46:59.889087 30133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:46:59.889358 30118 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260928 14:46:59.889554 30118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "056309a49057467982f9a1c646f19eff"
format_stamp: "Formatted at 2026-09-28 14:46:59 on dist-test-slave-pv0w"
server_key: "b60d8e174b7bacde24146d3fa1c8bd85"
server_key_iv: "3b7ca718e4bf96a1c12cd0ef4d1897f6"
server_key_version: "encryptionkey@0"
I20260928 14:46:59.889667 30118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:46:59.893322 30118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:46:59.895601 30118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:46:59.895809 30118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:46:59.899897 30118 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33241
I20260928 14:46:59.899945 30185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33241 every 8 connection(s)
I20260928 14:46:59.900334 30118 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:46:59.901358 30186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:46:59.903822 30186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 056309a49057467982f9a1c646f19eff: Bootstrap starting.
I20260928 14:46:59.904294 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 30118
I20260928 14:46:59.904436 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:46:59.904464 30186 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 056309a49057467982f9a1c646f19eff: Neither blocks nor log segments found. Creating new log.
I20260928 14:46:59.904649 9349 external_mini_cluster.cc:1428] Setting key 9c27a43d615186f40e3e47158be297af
I20260928 14:46:59.904893 30186 log.cc:824] T 00000000000000000000000000000000 P 056309a49057467982f9a1c646f19eff: Log is configured to *not* fsync() on all Append() calls
I20260928 14:46:59.905750 30186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 056309a49057467982f9a1c646f19eff: No bootstrap required, opened a new log
I20260928 14:46:59.907382 30190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:59.905225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54159 (local address 127.0.0.1:33241)
0928 14:46:59.905418 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:59.905427 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:59.905461 (+ 34us) server_negotiation.cc:407] Connection header received
0928 14:46:59.906060 (+ 599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:59.906069 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:59.906106 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:59.906275 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:59.906929 (+ 654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:59.906932 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:59.906933 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:46:59.906937 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:59.906984 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:46:59.907000 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:59.907090 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:59.907122 (+ 32us) server_negotiation.cc:299] Negotiation successful
0928 14:46:59.907176 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":83,"threads_started":1}
I20260928 14:46:59.908296 30186 raft_consensus.cc:348] T 00000000000000000000000000000000 P 056309a49057467982f9a1c646f19eff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "056309a49057467982f9a1c646f19eff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33241 } }
I20260928 14:46:59.908433 30186 raft_consensus.cc:374] T 00000000000000000000000000000000 P 056309a49057467982f9a1c646f19eff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:46:59.908491 30186 raft_consensus.cc:729] T 00000000000000000000000000000000 P 056309a49057467982f9a1c646f19eff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 056309a49057467982f9a1c646f19eff, State: Initialized, Role: FOLLOWER
I20260928 14:46:59.908536 9349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:46:59.908625 30186 consensus_queue.cc:260] T 00000000000000000000000000000000 P 056309a49057467982f9a1c646f19eff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "056309a49057467982f9a1c646f19eff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33241 } }
I20260928 14:46:59.908725 30186 raft_consensus.cc:388] T 00000000000000000000000000000000 P 056309a49057467982f9a1c646f19eff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:46:59.908753 30186 raft_consensus.cc:482] T 00000000000000000000000000000000 P 056309a49057467982f9a1c646f19eff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:46:59.908797 30186 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 056309a49057467982f9a1c646f19eff [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:46:59.909544 30186 raft_consensus.cc:504] T 00000000000000000000000000000000 P 056309a49057467982f9a1c646f19eff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "056309a49057467982f9a1c646f19eff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33241 } }
I20260928 14:46:59.909680 30186 leader_election.cc:302] T 00000000000000000000000000000000 P 056309a49057467982f9a1c646f19eff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 056309a49057467982f9a1c646f19eff; no voters:
I20260928 14:46:59.909871 30186 leader_election.cc:288] T 00000000000000000000000000000000 P 056309a49057467982f9a1c646f19eff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:46:59.909904 30195 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 056309a49057467982f9a1c646f19eff [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:46:59.910095 30195 raft_consensus.cc:686] T 00000000000000000000000000000000 P 056309a49057467982f9a1c646f19eff [term 1 LEADER]: Becoming Leader. State: Replica: 056309a49057467982f9a1c646f19eff, State: Running, Role: LEADER
I20260928 14:46:59.910117 30186 sys_catalog.cc:564] T 00000000000000000000000000000000 P 056309a49057467982f9a1c646f19eff [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:46:59.910843 30195 consensus_queue.cc:237] T 00000000000000000000000000000000 P 056309a49057467982f9a1c646f19eff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "056309a49057467982f9a1c646f19eff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33241 } }
I20260928 14:46:59.911386 30197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 056309a49057467982f9a1c646f19eff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 056309a49057467982f9a1c646f19eff. Latest consensus state: current_term: 1 leader_uuid: "056309a49057467982f9a1c646f19eff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "056309a49057467982f9a1c646f19eff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33241 } } }
I20260928 14:46:59.911487 30197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 056309a49057467982f9a1c646f19eff [sys.catalog]: This master's current role is: LEADER
I20260928 14:46:59.911862 30196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 056309a49057467982f9a1c646f19eff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "056309a49057467982f9a1c646f19eff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "056309a49057467982f9a1c646f19eff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33241 } } }
I20260928 14:46:59.912228 30190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:46:59.910395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34241 (local address 127.0.0.1:33241)
0928 14:46:59.910455 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0928 14:46:59.910458 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:46:59.910587 (+ 129us) server_negotiation.cc:407] Connection header received
0928 14:46:59.910693 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:46:59.910694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:46:59.910701 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:46:59.910770 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:46:59.911995 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:46:59.911997 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:46:59.911997 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:46:59.912002 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:46:59.912020 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:46:59.912025 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:46:59.912091 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0928 14:46:59.912152 (+ 61us) server_negotiation.cc:299] Negotiation successful
0928 14:46:59.912167 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260928 14:46:59.912325 30196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 056309a49057467982f9a1c646f19eff [sys.catalog]: This master's current role is: LEADER
W20260928 14:46:59.912376 30210 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 056309a49057467982f9a1c646f19eff: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:46:59.912423 30210 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:46:59.912468 30211 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:46:59.912616 30211 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:46:59.914137 30211 catalog_manager.cc:1347] Generated new cluster ID: 1b4d930fc151405d888802b07266c4e2
I20260928 14:46:59.914198 30211 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:46:59.926515 30211 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:46:59.926726 30211 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:46:59.932890 30211 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 056309a49057467982f9a1c646f19eff: Generated new TSK 0
I20260928 14:46:59.933115 30211 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:46:59.938019 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 30118
2026-09-28T14:46:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1316 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:46:59 dist-test-slave-pv0w krb5kdc[30218](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:46:59 dist-test-slave-pv0w krb5kdc[30218](info): set up 2 sockets
Sep 28 14:46:59 dist-test-slave-pv0w krb5kdc[30218](info): commencing operation
krb5kdc: starting...
W20260928 14:47:00.988744 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:47:01 dist-test-slave-pv0w krb5kdc[30218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:47:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:47:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 14:47:01.111012 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39573
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39573
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:47:01.193248 30231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:47:01.193491 30231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:47:01.193542 30231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:47:01.194723 30231 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:47:01.194788 30231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:47:01.194814 30231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:47:01.194847 30231 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:47:01.194873 30231 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:47:01.196395 30231 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39573
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:47:01.196717 30231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:47:01.196954 30231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:47:01.201988 30236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:47:01.201990 30239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:47:01.202082 30237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:47:01.202244 30231 server_base.cc:1034] running on GCE node
I20260928 14:47:01.202605 30231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:47:01.202936 30231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:47:01.204100 30231 hybrid_clock.cc:648] HybridClock initialized: now 1790606821204089 us; error 31 us; skew 500 ppm
Sep 28 14:47:01 dist-test-slave-pv0w krb5kdc[30218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606821, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:47:01.206898 30231 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 14:47:01.207348 30231 webserver.cc:460] Webserver started at http://127.0.0.1:38071/ using document root <none> and password file <none>
I20260928 14:47:01.207613 30231 fs_manager.cc:359] Metadata directory not provided
I20260928 14:47:01.207687 30231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:47:01.207893 30231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:47:01.208998 30231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "ff8673f5fd6b49c6a7de3f52107127f8"
format_stamp: "Formatted at 2026-09-28 14:47:01 on dist-test-slave-pv0w"
server_key: "9b03567e34137e66f413f2f2bfbfe99e"
server_key_iv: "63283247437fa49b4bbdfc4c155fb270"
server_key_version: "encryptionkey@0"
I20260928 14:47:01.209363 30231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "ff8673f5fd6b49c6a7de3f52107127f8"
format_stamp: "Formatted at 2026-09-28 14:47:01 on dist-test-slave-pv0w"
server_key: "9b03567e34137e66f413f2f2bfbfe99e"
server_key_iv: "63283247437fa49b4bbdfc4c155fb270"
server_key_version: "encryptionkey@0"
I20260928 14:47:01.210852 30231 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:47:01.211858 30246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:47:01.212082 30231 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:47:01.212194 30231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "ff8673f5fd6b49c6a7de3f52107127f8"
format_stamp: "Formatted at 2026-09-28 14:47:01 on dist-test-slave-pv0w"
server_key: "9b03567e34137e66f413f2f2bfbfe99e"
server_key_iv: "63283247437fa49b4bbdfc4c155fb270"
server_key_version: "encryptionkey@0"
I20260928 14:47:01.212271 30231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:47:01.234707 30231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:47:01.237301 30231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:47:01.237479 30231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:47:01.241436 30231 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39573
I20260928 14:47:01.241474 30298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39573 every 8 connection(s)
I20260928 14:47:01.241850 30231 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:47:01.242889 30299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:47:01.245277 30299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff8673f5fd6b49c6a7de3f52107127f8: Bootstrap starting.
I20260928 14:47:01.245875 30299 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff8673f5fd6b49c6a7de3f52107127f8: Neither blocks nor log segments found. Creating new log.
I20260928 14:47:01.246181 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 30231
I20260928 14:47:01.246209 30299 log.cc:824] T 00000000000000000000000000000000 P ff8673f5fd6b49c6a7de3f52107127f8: Log is configured to *not* fsync() on all Append() calls
I20260928 14:47:01.246312 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:47:01.246510 9349 external_mini_cluster.cc:1428] Setting key b1297c541e39544cde39d8d89595c3b4
I20260928 14:47:01.247020 30299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff8673f5fd6b49c6a7de3f52107127f8: No bootstrap required, opened a new log
I20260928 14:47:01.248793 30299 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff8673f5fd6b49c6a7de3f52107127f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff8673f5fd6b49c6a7de3f52107127f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39573 } }
I20260928 14:47:01.248931 30299 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff8673f5fd6b49c6a7de3f52107127f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:47:01.248955 30299 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff8673f5fd6b49c6a7de3f52107127f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff8673f5fd6b49c6a7de3f52107127f8, State: Initialized, Role: FOLLOWER
I20260928 14:47:01.249147 30299 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff8673f5fd6b49c6a7de3f52107127f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff8673f5fd6b49c6a7de3f52107127f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39573 } }
I20260928 14:47:01.249240 30299 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff8673f5fd6b49c6a7de3f52107127f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:47:01.249274 30299 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff8673f5fd6b49c6a7de3f52107127f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:47:01.249325 30299 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff8673f5fd6b49c6a7de3f52107127f8 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:47:01.250349 30299 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff8673f5fd6b49c6a7de3f52107127f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff8673f5fd6b49c6a7de3f52107127f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39573 } }
I20260928 14:47:01.250439 30299 leader_election.cc:302] T 00000000000000000000000000000000 P ff8673f5fd6b49c6a7de3f52107127f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff8673f5fd6b49c6a7de3f52107127f8; no voters:
I20260928 14:47:01.250653 30299 leader_election.cc:288] T 00000000000000000000000000000000 P ff8673f5fd6b49c6a7de3f52107127f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:47:01.250679 30304 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff8673f5fd6b49c6a7de3f52107127f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:47:01.250789 30304 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff8673f5fd6b49c6a7de3f52107127f8 [term 1 LEADER]: Becoming Leader. State: Replica: ff8673f5fd6b49c6a7de3f52107127f8, State: Running, Role: LEADER
I20260928 14:47:01.250926 30299 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff8673f5fd6b49c6a7de3f52107127f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:47:01.250929 30304 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff8673f5fd6b49c6a7de3f52107127f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff8673f5fd6b49c6a7de3f52107127f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39573 } }
I20260928 14:47:01.251360 30305 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff8673f5fd6b49c6a7de3f52107127f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff8673f5fd6b49c6a7de3f52107127f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff8673f5fd6b49c6a7de3f52107127f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39573 } } }
I20260928 14:47:01.251454 30305 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff8673f5fd6b49c6a7de3f52107127f8 [sys.catalog]: This master's current role is: LEADER
I20260928 14:47:01.251374 30306 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff8673f5fd6b49c6a7de3f52107127f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff8673f5fd6b49c6a7de3f52107127f8. Latest consensus state: current_term: 1 leader_uuid: "ff8673f5fd6b49c6a7de3f52107127f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff8673f5fd6b49c6a7de3f52107127f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39573 } } }
I20260928 14:47:01.251641 30306 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff8673f5fd6b49c6a7de3f52107127f8 [sys.catalog]: This master's current role is: LEADER
I20260928 14:47:01.252606 30311 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:47:01.253170 30311 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:47:01.254781 30311 catalog_manager.cc:1347] Generated new cluster ID: f55584d9f13c4cdabddbab5facf97fdb
I20260928 14:47:01.254832 30311 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 14:47:01 dist-test-slave-pv0w krb5kdc[30218](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260928 14:47:01.259967 30303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:01.247053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.54:53017 (local address 127.0.0.1:39573)
0928 14:47:01.247214 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:01.247221 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:01.247248 (+ 27us) server_negotiation.cc:407] Connection header received
0928 14:47:01.248796 (+ 1548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:01.248804 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:01.248861 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:01.249034 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:01.249928 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:01.251156 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:01.252239 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:01.252868 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:01.252997 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:01.256006 (+ 3009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:01.256011 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:01.256011 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:47:01.256023 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:01.258014 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:01.258479 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:01.258482 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:01.258483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:01.258607 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:01.258858 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:01.258859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:01.258859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:01.259181 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:01.259333 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:01.259480 (+ 147us) server_negotiation.cc:299] Negotiation successful
0928 14:47:01.259578 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":67,"threads_started":1}
I20260928 14:47:01.261003 9349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:47:01.271123 30303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:01.262504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.54:52033 (local address 127.0.0.1:39573)
0928 14:47:01.262541 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:01.262543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:01.263880 (+ 1337us) server_negotiation.cc:407] Connection header received
0928 14:47:01.263913 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:01.263914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:01.263927 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:01.263999 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:01.264577 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:01.265362 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:01.266343 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:01.266711 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:01.266784 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:01.268201 (+ 1417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:01.268202 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:01.268203 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:47:01.268210 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:01.269848 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:01.270275 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:01.270276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:01.270277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:01.270338 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:01.270564 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:01.270565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:01.270565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:01.270705 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:01.270810 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:01.270943 (+ 133us) server_negotiation.cc:299] Negotiation successful
0928 14:47:01.271010 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 14:47:01.272643 30311 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:47:01.273386 30311 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:47:01.290978 30311 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff8673f5fd6b49c6a7de3f52107127f8: Generated new TSK 0
I20260928 14:47:01.291188 30311 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:47:01.299005 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 30231
W20260928 14:47:01.303507 30216 connection.cc:535] client connection to 127.0.0.1:39573 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39573): Connection reset by peer (error 104)
2026-09-28T14:47:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1360 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:47:01 dist-test-slave-pv0w krb5kdc[30330](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:47:01 dist-test-slave-pv0w krb5kdc[30330](info): set up 2 sockets
Sep 28 14:47:01 dist-test-slave-pv0w krb5kdc[30330](info): commencing operation
krb5kdc: starting...
W20260928 14:47:02.352734 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:47:02 dist-test-slave-pv0w krb5kdc[30330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606822, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:47:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:47:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 14:47:02.479095 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40507
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40507
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:47:02.562000 30343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:47:02.562235 30343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:47:02.562261 30343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:47:02.563491 30343 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:47:02.563544 30343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:47:02.563560 30343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:47:02.563575 30343 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:47:02.563587 30343 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:47:02.565194 30343 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40507
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:47:02.565454 30343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:47:02.565722 30343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:47:02.571029 30349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:47:02.571038 30348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:47:02.571038 30351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:47:02.571226 30343 server_base.cc:1034] running on GCE node
I20260928 14:47:02.571488 30343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:47:02.571839 30343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:47:02.573004 30343 hybrid_clock.cc:648] HybridClock initialized: now 1790606822572980 us; error 39 us; skew 500 ppm
Sep 28 14:47:02 dist-test-slave-pv0w krb5kdc[30330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606822, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:47:02.575680 30343 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 14:47:02.576155 30343 webserver.cc:460] Webserver started at http://127.0.0.1:45357/ using document root <none> and password file <none>
I20260928 14:47:02.576366 30343 fs_manager.cc:359] Metadata directory not provided
I20260928 14:47:02.576407 30343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:47:02.576495 30343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:47:02.577535 30343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "5f36225949f94b20b009da6477e2f0fb"
format_stamp: "Formatted at 2026-09-28 14:47:02 on dist-test-slave-pv0w"
server_key: "40d2d613dbedfc5c6329da18f7ba7970"
server_key_iv: "63be3ba9888e018bbb3c3112495694a9"
server_key_version: "encryptionkey@0"
I20260928 14:47:02.577845 30343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "5f36225949f94b20b009da6477e2f0fb"
format_stamp: "Formatted at 2026-09-28 14:47:02 on dist-test-slave-pv0w"
server_key: "40d2d613dbedfc5c6329da18f7ba7970"
server_key_iv: "63be3ba9888e018bbb3c3112495694a9"
server_key_version: "encryptionkey@0"
I20260928 14:47:02.579159 30343 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:47:02.579947 30358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:47:02.580130 30343 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:47:02.580246 30343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "5f36225949f94b20b009da6477e2f0fb"
format_stamp: "Formatted at 2026-09-28 14:47:02 on dist-test-slave-pv0w"
server_key: "40d2d613dbedfc5c6329da18f7ba7970"
server_key_iv: "63be3ba9888e018bbb3c3112495694a9"
server_key_version: "encryptionkey@0"
I20260928 14:47:02.580329 30343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:47:02.591630 30343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:47:02.594164 30343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:47:02.594363 30343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:47:02.598793 30343 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40507
I20260928 14:47:02.598830 30410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40507 every 8 connection(s)
I20260928 14:47:02.599211 30343 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:47:02.600334 30411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:47:02.602684 30411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f36225949f94b20b009da6477e2f0fb: Bootstrap starting.
I20260928 14:47:02.603297 30411 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f36225949f94b20b009da6477e2f0fb: Neither blocks nor log segments found. Creating new log.
I20260928 14:47:02.603600 30411 log.cc:824] T 00000000000000000000000000000000 P 5f36225949f94b20b009da6477e2f0fb: Log is configured to *not* fsync() on all Append() calls
I20260928 14:47:02.604250 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 30343
I20260928 14:47:02.604379 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:47:02.604508 30411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f36225949f94b20b009da6477e2f0fb: No bootstrap required, opened a new log
I20260928 14:47:02.604609 9349 external_mini_cluster.cc:1428] Setting key 6af8fc39f1c7d6764903f032dd90535a
I20260928 14:47:02.607040 30411 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f36225949f94b20b009da6477e2f0fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f36225949f94b20b009da6477e2f0fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40507 } }
I20260928 14:47:02.607205 30411 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f36225949f94b20b009da6477e2f0fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:47:02.607228 30411 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f36225949f94b20b009da6477e2f0fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f36225949f94b20b009da6477e2f0fb, State: Initialized, Role: FOLLOWER
I20260928 14:47:02.607378 30411 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f36225949f94b20b009da6477e2f0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f36225949f94b20b009da6477e2f0fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40507 } }
I20260928 14:47:02.607477 30411 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f36225949f94b20b009da6477e2f0fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:47:02.607524 30411 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f36225949f94b20b009da6477e2f0fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:47:02.607599 30411 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f36225949f94b20b009da6477e2f0fb [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:47:02.608299 30411 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f36225949f94b20b009da6477e2f0fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f36225949f94b20b009da6477e2f0fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40507 } }
I20260928 14:47:02.608434 30411 leader_election.cc:302] T 00000000000000000000000000000000 P 5f36225949f94b20b009da6477e2f0fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f36225949f94b20b009da6477e2f0fb; no voters:
I20260928 14:47:02.608662 30411 leader_election.cc:288] T 00000000000000000000000000000000 P 5f36225949f94b20b009da6477e2f0fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:47:02.608692 30416 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f36225949f94b20b009da6477e2f0fb [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:47:02.608801 30416 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f36225949f94b20b009da6477e2f0fb [term 1 LEADER]: Becoming Leader. State: Replica: 5f36225949f94b20b009da6477e2f0fb, State: Running, Role: LEADER
I20260928 14:47:02.608945 30416 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f36225949f94b20b009da6477e2f0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f36225949f94b20b009da6477e2f0fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40507 } }
I20260928 14:47:02.608950 30411 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f36225949f94b20b009da6477e2f0fb [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:47:02.610497 30415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:02.605196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.54:46263 (local address 127.0.0.1:40507)
0928 14:47:02.605396 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:02.605412 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:02.605439 (+ 27us) server_negotiation.cc:407] Connection header received
0928 14:47:02.605886 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:02.605892 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:02.605947 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:02.606102 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:02.606351 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:02.607360 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:02.608258 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:02.608861 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:02.609351 (+ 490us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:02.609827 (+ 476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:02.609831 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:02.609832 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:47:02.610028 (+ 196us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:02.610060 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:47:02.610072 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:02.610171 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:02.610257 (+ 86us) server_negotiation.cc:299] Negotiation successful
0928 14:47:02.610286 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":77,"threads_started":1}
I20260928 14:47:02.611011 30418 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f36225949f94b20b009da6477e2f0fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f36225949f94b20b009da6477e2f0fb. Latest consensus state: current_term: 1 leader_uuid: "5f36225949f94b20b009da6477e2f0fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f36225949f94b20b009da6477e2f0fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40507 } } }
I20260928 14:47:02.611281 30418 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f36225949f94b20b009da6477e2f0fb [sys.catalog]: This master's current role is: LEADER
I20260928 14:47:02.611151 30417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f36225949f94b20b009da6477e2f0fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f36225949f94b20b009da6477e2f0fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f36225949f94b20b009da6477e2f0fb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40507 } } }
I20260928 14:47:02.611405 30417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f36225949f94b20b009da6477e2f0fb [sys.catalog]: This master's current role is: LEADER
W20260928 14:47:02.611435 30430 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5f36225949f94b20b009da6477e2f0fb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:47:02.611474 30430 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:47:02.611533 9349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:47:02.611608 30432 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:47:02.611825 30432 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:47:02.613623 30432 catalog_manager.cc:1347] Generated new cluster ID: 92170aa9935b42b29ea4657b8946c1c3
I20260928 14:47:02.613708 30432 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:47:02.616825 30415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:02.613174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.54:48805 (local address 127.0.0.1:40507)
0928 14:47:02.613213 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:02.613215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:02.613313 (+ 98us) server_negotiation.cc:407] Connection header received
0928 14:47:02.613362 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:02.613363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:02.613373 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:02.613447 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:02.613602 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:02.614317 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:02.615335 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:02.615694 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:02.615796 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:02.616574 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:02.616575 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:02.616576 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:47:02.616581 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:02.616599 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:47:02.616604 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:02.616669 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:02.616737 (+ 68us) server_negotiation.cc:299] Negotiation successful
0928 14:47:02.616748 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 14:47:02.620970 30432 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:47:02.621464 30432 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:47:02.632827 30432 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f36225949f94b20b009da6477e2f0fb: Generated new TSK 0
I20260928 14:47:02.633072 30432 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:47:02.635850 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 30343
W20260928 14:47:02.640431 30328 connection.cc:535] client connection to 127.0.0.1:40507 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40507): Connection reset by peer (error 104)
2026-09-28T14:47:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1336 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:47:02 dist-test-slave-pv0w krb5kdc[30442](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:47:02 dist-test-slave-pv0w krb5kdc[30442](info): set up 2 sockets
Sep 28 14:47:02 dist-test-slave-pv0w krb5kdc[30442](info): commencing operation
krb5kdc: starting...
W20260928 14:47:03.668653 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:47:03 dist-test-slave-pv0w krb5kdc[30442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606823, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:47:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:47:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 14:47:03.794426 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34247
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34247
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:47:03.878597 30455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:47:03.878820 30455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:47:03.878845 30455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:47:03.880087 30455 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:47:03.880134 30455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:47:03.880149 30455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:47:03.880162 30455 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:47:03.880218 30455 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:47:03.881738 30455 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:47:03.882015 30455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:47:03.882261 30455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260928 14:47:03.887511 30455 server_base.cc:1034] running on GCE node
W20260928 14:47:03.887480 30461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:47:03.887470 30463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:47:03.887480 30460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:47:03.888025 30455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:47:03.888368 30455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:47:03.889576 30455 hybrid_clock.cc:648] HybridClock initialized: now 1790606823889553 us; error 41 us; skew 500 ppm
Sep 28 14:47:03 dist-test-slave-pv0w krb5kdc[30442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606823, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:47:03.892346 30455 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 14:47:03.892812 30455 webserver.cc:460] Webserver started at http://127.0.0.1:39573/ using document root <none> and password file <none>
I20260928 14:47:03.893034 30455 fs_manager.cc:359] Metadata directory not provided
I20260928 14:47:03.893078 30455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:47:03.893170 30455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:47:03.894207 30455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "f2cd3f3e19b6438fabf60801078cff85"
format_stamp: "Formatted at 2026-09-28 14:47:03 on dist-test-slave-pv0w"
server_key: "fcf0f1587e43180359e2a0456c5b9d99"
server_key_iv: "b38f6b6311745d5fc627fa730b9c3e02"
server_key_version: "encryptionkey@0"
I20260928 14:47:03.894515 30455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "f2cd3f3e19b6438fabf60801078cff85"
format_stamp: "Formatted at 2026-09-28 14:47:03 on dist-test-slave-pv0w"
server_key: "fcf0f1587e43180359e2a0456c5b9d99"
server_key_iv: "b38f6b6311745d5fc627fa730b9c3e02"
server_key_version: "encryptionkey@0"
I20260928 14:47:03.895879 30455 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260928 14:47:03.896662 30470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:47:03.896929 30455 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:47:03.897055 30455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "f2cd3f3e19b6438fabf60801078cff85"
format_stamp: "Formatted at 2026-09-28 14:47:03 on dist-test-slave-pv0w"
server_key: "fcf0f1587e43180359e2a0456c5b9d99"
server_key_iv: "b38f6b6311745d5fc627fa730b9c3e02"
server_key_version: "encryptionkey@0"
I20260928 14:47:03.897140 30455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:47:03.901100 30455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:47:03.903102 30455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:47:03.903263 30455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:47:03.907405 30455 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34247
I20260928 14:47:03.907493 30522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34247 every 8 connection(s)
I20260928 14:47:03.907883 30455 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:47:03.908885 30523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:47:03.909299 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 30455
I20260928 14:47:03.909423 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:47:03.909658 9349 external_mini_cluster.cc:1428] Setting key d6dadb725469322973c88a6f4671b7b3
I20260928 14:47:03.912065 30523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2cd3f3e19b6438fabf60801078cff85: Bootstrap starting.
I20260928 14:47:03.912533 30526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:03.910265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.54:50465 (local address 127.0.0.1:34247)
0928 14:47:03.910473 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:03.910483 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:03.910517 (+ 34us) server_negotiation.cc:407] Connection header received
0928 14:47:03.911081 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:03.911090 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:03.911128 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:03.911284 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:03.912038 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:03.912042 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:03.912042 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:47:03.912046 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:03.912099 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:47:03.912114 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:03.912190 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:03.912272 (+ 82us) server_negotiation.cc:299] Negotiation successful
0928 14:47:03.912315 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":92,"threads_started":1}
I20260928 14:47:03.912784 30523 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2cd3f3e19b6438fabf60801078cff85: Neither blocks nor log segments found. Creating new log.
I20260928 14:47:03.913072 30523 log.cc:824] T 00000000000000000000000000000000 P f2cd3f3e19b6438fabf60801078cff85: Log is configured to *not* fsync() on all Append() calls
I20260928 14:47:03.913622 9349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:47:03.913968 30523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2cd3f3e19b6438fabf60801078cff85: No bootstrap required, opened a new log
I20260928 14:47:03.916297 30526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:03.915074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.54:49379 (local address 127.0.0.1:34247)
0928 14:47:03.915137 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:03.915139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:03.915220 (+ 81us) server_negotiation.cc:407] Connection header received
0928 14:47:03.915282 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:03.915283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:03.915288 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:03.915357 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:03.916032 (+ 675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:03.916033 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:03.916034 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:47:03.916038 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:03.916061 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:47:03.916065 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:03.916132 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:03.916167 (+ 35us) server_negotiation.cc:299] Negotiation successful
0928 14:47:03.916182 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260928 14:47:03.916185 30523 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2cd3f3e19b6438fabf60801078cff85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2cd3f3e19b6438fabf60801078cff85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34247 } }
I20260928 14:47:03.916385 30523 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2cd3f3e19b6438fabf60801078cff85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:47:03.916424 30523 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2cd3f3e19b6438fabf60801078cff85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2cd3f3e19b6438fabf60801078cff85, State: Initialized, Role: FOLLOWER
I20260928 14:47:03.916723 30523 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2cd3f3e19b6438fabf60801078cff85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2cd3f3e19b6438fabf60801078cff85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34247 } }
I20260928 14:47:03.916837 30523 raft_consensus.cc:388] T 00000000000000000000000000000000 P f2cd3f3e19b6438fabf60801078cff85 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:47:03.916885 30523 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2cd3f3e19b6438fabf60801078cff85 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:47:03.916940 30523 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2cd3f3e19b6438fabf60801078cff85 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:47:03.917510 30523 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2cd3f3e19b6438fabf60801078cff85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2cd3f3e19b6438fabf60801078cff85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34247 } }
I20260928 14:47:03.917641 30523 leader_election.cc:302] T 00000000000000000000000000000000 P f2cd3f3e19b6438fabf60801078cff85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f2cd3f3e19b6438fabf60801078cff85; no voters:
I20260928 14:47:03.917830 30523 leader_election.cc:288] T 00000000000000000000000000000000 P f2cd3f3e19b6438fabf60801078cff85 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:47:03.917887 30533 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2cd3f3e19b6438fabf60801078cff85 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:47:03.918063 30533 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2cd3f3e19b6438fabf60801078cff85 [term 1 LEADER]: Becoming Leader. State: Replica: f2cd3f3e19b6438fabf60801078cff85, State: Running, Role: LEADER
I20260928 14:47:03.918087 30523 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2cd3f3e19b6438fabf60801078cff85 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:47:03.918155 30533 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2cd3f3e19b6438fabf60801078cff85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2cd3f3e19b6438fabf60801078cff85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34247 } }
I20260928 14:47:03.918543 30535 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2cd3f3e19b6438fabf60801078cff85 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2cd3f3e19b6438fabf60801078cff85. Latest consensus state: current_term: 1 leader_uuid: "f2cd3f3e19b6438fabf60801078cff85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2cd3f3e19b6438fabf60801078cff85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34247 } } }
I20260928 14:47:03.918790 30535 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2cd3f3e19b6438fabf60801078cff85 [sys.catalog]: This master's current role is: LEADER
I20260928 14:47:03.918526 30534 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2cd3f3e19b6438fabf60801078cff85 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f2cd3f3e19b6438fabf60801078cff85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2cd3f3e19b6438fabf60801078cff85" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34247 } } }
I20260928 14:47:03.918991 30540 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:47:03.919075 30534 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2cd3f3e19b6438fabf60801078cff85 [sys.catalog]: This master's current role is: LEADER
I20260928 14:47:03.919636 30540 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:47:03.921217 30540 catalog_manager.cc:1347] Generated new cluster ID: d3b52ebb6a414dd3b79f1356968f2954
I20260928 14:47:03.921269 30540 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:47:03.943763 30540 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:47:03.943969 30540 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:47:03.947402 30540 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f2cd3f3e19b6438fabf60801078cff85: Generated new TSK 0
I20260928 14:47:03.947613 30540 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:47:03.950484 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 30455
2026-09-28T14:47:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1314 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:47:04 dist-test-slave-pv0w krb5kdc[30554](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:47:04 dist-test-slave-pv0w krb5kdc[30554](info): set up 2 sockets
Sep 28 14:47:04 dist-test-slave-pv0w krb5kdc[30554](info): commencing operation
krb5kdc: starting...
W20260928 14:47:05.012707 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:47:05 dist-test-slave-pv0w krb5kdc[30554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606825, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:47:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:47:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 14:47:05.139298 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35589
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35589
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:47:05.221109 30567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:47:05.221387 30567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:47:05.221428 30567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:47:05.222631 30567 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:47:05.222714 30567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:47:05.222738 30567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:47:05.222760 30567 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:47:05.222790 30567 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:47:05.224368 30567 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35589
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:47:05.224663 30567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:47:05.224915 30567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:47:05.230187 30575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:47:05.230268 30572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:47:05.230355 30567 server_base.cc:1034] running on GCE node
W20260928 14:47:05.230268 30573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:47:05.230844 30567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:47:05.231182 30567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:47:05.232379 30567 hybrid_clock.cc:648] HybridClock initialized: now 1790606825232372 us; error 59 us; skew 500 ppm
Sep 28 14:47:05 dist-test-slave-pv0w krb5kdc[30554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606825, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:47:05.239401 30567 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 14:47:05.240039 30567 webserver.cc:460] Webserver started at http://127.0.0.1:45375/ using document root <none> and password file <none>
I20260928 14:47:05.240315 30567 fs_manager.cc:359] Metadata directory not provided
I20260928 14:47:05.240387 30567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:47:05.240549 30567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:47:05.241688 30567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "ad30c5adcf0741db867ae4a43279e368"
format_stamp: "Formatted at 2026-09-28 14:47:05 on dist-test-slave-pv0w"
server_key: "097921c5760f3716a9e9d46d3d836cb4"
server_key_iv: "25789a326d25955058a28597fd0bb1ba"
server_key_version: "encryptionkey@0"
I20260928 14:47:05.242062 30567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "ad30c5adcf0741db867ae4a43279e368"
format_stamp: "Formatted at 2026-09-28 14:47:05 on dist-test-slave-pv0w"
server_key: "097921c5760f3716a9e9d46d3d836cb4"
server_key_iv: "25789a326d25955058a28597fd0bb1ba"
server_key_version: "encryptionkey@0"
I20260928 14:47:05.243455 30567 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:47:05.244367 30582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:47:05.244599 30567 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260928 14:47:05.244714 30567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "ad30c5adcf0741db867ae4a43279e368"
format_stamp: "Formatted at 2026-09-28 14:47:05 on dist-test-slave-pv0w"
server_key: "097921c5760f3716a9e9d46d3d836cb4"
server_key_iv: "25789a326d25955058a28597fd0bb1ba"
server_key_version: "encryptionkey@0"
I20260928 14:47:05.244796 30567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:47:05.257601 30567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:47:05.259931 30567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:47:05.260104 30567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:47:05.264278 30567 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35589
I20260928 14:47:05.264318 30634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35589 every 8 connection(s)
I20260928 14:47:05.264681 30567 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:47:05.265693 30635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:47:05.268451 30635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad30c5adcf0741db867ae4a43279e368: Bootstrap starting.
I20260928 14:47:05.269143 30635 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad30c5adcf0741db867ae4a43279e368: Neither blocks nor log segments found. Creating new log.
I20260928 14:47:05.269467 30635 log.cc:824] T 00000000000000000000000000000000 P ad30c5adcf0741db867ae4a43279e368: Log is configured to *not* fsync() on all Append() calls
I20260928 14:47:05.270272 30635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad30c5adcf0741db867ae4a43279e368: No bootstrap required, opened a new log
I20260928 14:47:05.272094 30635 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad30c5adcf0741db867ae4a43279e368 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad30c5adcf0741db867ae4a43279e368" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35589 } }
I20260928 14:47:05.272269 30635 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad30c5adcf0741db867ae4a43279e368 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:47:05.272317 30635 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad30c5adcf0741db867ae4a43279e368 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad30c5adcf0741db867ae4a43279e368, State: Initialized, Role: FOLLOWER
I20260928 14:47:05.272475 30635 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad30c5adcf0741db867ae4a43279e368 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad30c5adcf0741db867ae4a43279e368" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35589 } }
I20260928 14:47:05.272558 30635 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad30c5adcf0741db867ae4a43279e368 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:47:05.272603 30635 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad30c5adcf0741db867ae4a43279e368 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:47:05.272662 30635 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad30c5adcf0741db867ae4a43279e368 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:47:05.273233 30635 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad30c5adcf0741db867ae4a43279e368 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad30c5adcf0741db867ae4a43279e368" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35589 } }
I20260928 14:47:05.273358 30635 leader_election.cc:302] T 00000000000000000000000000000000 P ad30c5adcf0741db867ae4a43279e368 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad30c5adcf0741db867ae4a43279e368; no voters:
I20260928 14:47:05.273562 30635 leader_election.cc:288] T 00000000000000000000000000000000 P ad30c5adcf0741db867ae4a43279e368 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:47:05.273612 30638 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad30c5adcf0741db867ae4a43279e368 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:47:05.273840 30638 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad30c5adcf0741db867ae4a43279e368 [term 1 LEADER]: Becoming Leader. State: Replica: ad30c5adcf0741db867ae4a43279e368, State: Running, Role: LEADER
I20260928 14:47:05.273844 30635 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad30c5adcf0741db867ae4a43279e368 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:47:05.273986 30638 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad30c5adcf0741db867ae4a43279e368 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad30c5adcf0741db867ae4a43279e368" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35589 } }
I20260928 14:47:05.274573 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 30567
I20260928 14:47:05.274713 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:47:05.274920 9349 external_mini_cluster.cc:1428] Setting key 23530bef5c251d3c83c3fe4717a9469e
I20260928 14:47:05.275406 30639 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad30c5adcf0741db867ae4a43279e368 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad30c5adcf0741db867ae4a43279e368" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad30c5adcf0741db867ae4a43279e368" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35589 } } }
I20260928 14:47:05.275738 30639 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad30c5adcf0741db867ae4a43279e368 [sys.catalog]: This master's current role is: LEADER
I20260928 14:47:05.275877 30640 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad30c5adcf0741db867ae4a43279e368 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad30c5adcf0741db867ae4a43279e368. Latest consensus state: current_term: 1 leader_uuid: "ad30c5adcf0741db867ae4a43279e368" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad30c5adcf0741db867ae4a43279e368" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35589 } } }
I20260928 14:47:05.275967 30640 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad30c5adcf0741db867ae4a43279e368 [sys.catalog]: This master's current role is: LEADER
W20260928 14:47:05.276369 30653 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ad30c5adcf0741db867ae4a43279e368: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:47:05.276470 30653 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:47:05.276542 30656 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:47:05.276723 30656 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:47:05.278303 30656 catalog_manager.cc:1347] Generated new cluster ID: f283ba139c3a4546b723e858b8696d32
I20260928 14:47:05.278355 30656 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 14:47:05 dist-test-slave-pv0w krb5kdc[30554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606825, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260928 14:47:05.287477 30656 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:47:05.287513 30655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:05.275403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.54:48485 (local address 127.0.0.1:35589)
0928 14:47:05.275934 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:05.275944 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:05.275994 (+ 50us) server_negotiation.cc:407] Connection header received
0928 14:47:05.276123 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:05.276129 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:05.276206 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:05.276362 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:05.277238 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:05.278637 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:05.279705 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:05.280389 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:05.280510 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:05.283560 (+ 3050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:05.283565 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:05.283565 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:47:05.283579 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:05.285593 (+ 2014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:05.286088 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:05.286091 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:05.286091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:05.286225 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:05.286494 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:05.286495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:05.286495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:05.286807 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:05.286966 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:05.287133 (+ 167us) server_negotiation.cc:299] Negotiation successful
0928 14:47:05.287258 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":87,"threads_started":1}
I20260928 14:47:05.288329 30656 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:47:05.288663 9349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:47:05.297832 30655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:05.290211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.54:40387 (local address 127.0.0.1:35589)
0928 14:47:05.290248 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:05.290250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:05.290325 (+ 75us) server_negotiation.cc:407] Connection header received
0928 14:47:05.290379 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:05.290380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:05.290390 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:05.290456 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:05.291039 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:05.291900 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:05.292928 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:05.293273 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:05.293346 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:05.294743 (+ 1397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:05.294745 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:05.294745 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:47:05.294752 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:05.296547 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:05.296978 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:05.296980 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:05.296980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:05.297052 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:05.297281 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:05.297282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:05.297282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:05.297418 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:05.297526 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:05.297662 (+ 136us) server_negotiation.cc:299] Negotiation successful
0928 14:47:05.297729 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 14:47:05.298647 30656 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad30c5adcf0741db867ae4a43279e368: Generated new TSK 0
I20260928 14:47:05.298851 30656 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:47:05.303786 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 30567
W20260928 14:47:05.307858 30552 connection.cc:535] client connection to 127.0.0.1:35589 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35589): Connection reset by peer (error 104)
2026-09-28T14:47:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1352 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:47:05 dist-test-slave-pv0w krb5kdc[30666](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:47:05 dist-test-slave-pv0w krb5kdc[30666](info): set up 2 sockets
Sep 28 14:47:05 dist-test-slave-pv0w krb5kdc[30666](info): commencing operation
krb5kdc: starting...
W20260928 14:47:06.348630 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:47:06 dist-test-slave-pv0w krb5kdc[30666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606826, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:47:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:47:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 14:47:06.475212 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46715
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:47:06.558184 30679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:47:06.558403 30679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:47:06.558429 30679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:47:06.559644 30679 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:47:06.559692 30679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:47:06.559707 30679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:47:06.559775 30679 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:47:06.559798 30679 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:47:06.561343 30679 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46715
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:47:06.561558 30679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:47:06.561739 30679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:47:06.566793 30687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:47:06.566794 30684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:47:06.567091 30679 server_base.cc:1034] running on GCE node
W20260928 14:47:06.566793 30685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:47:06.567533 30679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:47:06.567890 30679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:47:06.569078 30679 hybrid_clock.cc:648] HybridClock initialized: now 1790606826569051 us; error 47 us; skew 500 ppm
Sep 28 14:47:06 dist-test-slave-pv0w krb5kdc[30666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606826, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:47:06.571830 30679 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 14:47:06.572357 30679 webserver.cc:460] Webserver started at http://127.0.0.1:38497/ using document root <none> and password file <none>
I20260928 14:47:06.572598 30679 fs_manager.cc:359] Metadata directory not provided
I20260928 14:47:06.572670 30679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:47:06.572836 30679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:47:06.573961 30679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "e700f543e3034849a0487b6c0e205741"
format_stamp: "Formatted at 2026-09-28 14:47:06 on dist-test-slave-pv0w"
server_key: "5d290029976bb47b4f17845c0d2343f2"
server_key_iv: "9516a9a287f7f07dad09cc20456db558"
server_key_version: "encryptionkey@0"
I20260928 14:47:06.574342 30679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "e700f543e3034849a0487b6c0e205741"
format_stamp: "Formatted at 2026-09-28 14:47:06 on dist-test-slave-pv0w"
server_key: "5d290029976bb47b4f17845c0d2343f2"
server_key_iv: "9516a9a287f7f07dad09cc20456db558"
server_key_version: "encryptionkey@0"
I20260928 14:47:06.575804 30679 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:47:06.576608 30694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:47:06.576804 30679 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:47:06.576910 30679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "e700f543e3034849a0487b6c0e205741"
format_stamp: "Formatted at 2026-09-28 14:47:06 on dist-test-slave-pv0w"
server_key: "5d290029976bb47b4f17845c0d2343f2"
server_key_iv: "9516a9a287f7f07dad09cc20456db558"
server_key_version: "encryptionkey@0"
I20260928 14:47:06.577003 30679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:47:06.594949 30679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:47:06.597177 30679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:47:06.597560 30679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:47:06.602438 30679 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46715
I20260928 14:47:06.602480 30746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46715 every 8 connection(s)
I20260928 14:47:06.602936 30679 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:47:06.603984 30747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:47:06.606490 30747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e700f543e3034849a0487b6c0e205741: Bootstrap starting.
I20260928 14:47:06.607133 30747 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e700f543e3034849a0487b6c0e205741: Neither blocks nor log segments found. Creating new log.
I20260928 14:47:06.607432 30747 log.cc:824] T 00000000000000000000000000000000 P e700f543e3034849a0487b6c0e205741: Log is configured to *not* fsync() on all Append() calls
I20260928 14:47:06.608373 30747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e700f543e3034849a0487b6c0e205741: No bootstrap required, opened a new log
I20260928 14:47:06.610239 30747 raft_consensus.cc:348] T 00000000000000000000000000000000 P e700f543e3034849a0487b6c0e205741 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e700f543e3034849a0487b6c0e205741" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46715 } }
I20260928 14:47:06.610464 30747 raft_consensus.cc:374] T 00000000000000000000000000000000 P e700f543e3034849a0487b6c0e205741 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:47:06.610520 30747 raft_consensus.cc:729] T 00000000000000000000000000000000 P e700f543e3034849a0487b6c0e205741 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e700f543e3034849a0487b6c0e205741, State: Initialized, Role: FOLLOWER
I20260928 14:47:06.610605 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 30679
I20260928 14:47:06.610724 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:47:06.610693 30747 consensus_queue.cc:260] T 00000000000000000000000000000000 P e700f543e3034849a0487b6c0e205741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e700f543e3034849a0487b6c0e205741" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46715 } }
I20260928 14:47:06.610786 30747 raft_consensus.cc:388] T 00000000000000000000000000000000 P e700f543e3034849a0487b6c0e205741 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:47:06.610821 30747 raft_consensus.cc:482] T 00000000000000000000000000000000 P e700f543e3034849a0487b6c0e205741 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:47:06.610877 30747 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e700f543e3034849a0487b6c0e205741 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:47:06.610934 9349 external_mini_cluster.cc:1428] Setting key 77032a03bd419e51653dae76270969d8
I20260928 14:47:06.611673 30747 raft_consensus.cc:504] T 00000000000000000000000000000000 P e700f543e3034849a0487b6c0e205741 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e700f543e3034849a0487b6c0e205741" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46715 } }
I20260928 14:47:06.611866 30747 leader_election.cc:302] T 00000000000000000000000000000000 P e700f543e3034849a0487b6c0e205741 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e700f543e3034849a0487b6c0e205741; no voters:
I20260928 14:47:06.612067 30747 leader_election.cc:288] T 00000000000000000000000000000000 P e700f543e3034849a0487b6c0e205741 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:47:06.612088 30752 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e700f543e3034849a0487b6c0e205741 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:47:06.612207 30752 raft_consensus.cc:686] T 00000000000000000000000000000000 P e700f543e3034849a0487b6c0e205741 [term 1 LEADER]: Becoming Leader. State: Replica: e700f543e3034849a0487b6c0e205741, State: Running, Role: LEADER
I20260928 14:47:06.612357 30747 sys_catalog.cc:564] T 00000000000000000000000000000000 P e700f543e3034849a0487b6c0e205741 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:47:06.612380 30752 consensus_queue.cc:237] T 00000000000000000000000000000000 P e700f543e3034849a0487b6c0e205741 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e700f543e3034849a0487b6c0e205741" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46715 } }
I20260928 14:47:06.612776 30752 sys_catalog.cc:455] T 00000000000000000000000000000000 P e700f543e3034849a0487b6c0e205741 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e700f543e3034849a0487b6c0e205741. Latest consensus state: current_term: 1 leader_uuid: "e700f543e3034849a0487b6c0e205741" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e700f543e3034849a0487b6c0e205741" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46715 } } }
I20260928 14:47:06.612789 30753 sys_catalog.cc:455] T 00000000000000000000000000000000 P e700f543e3034849a0487b6c0e205741 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e700f543e3034849a0487b6c0e205741" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e700f543e3034849a0487b6c0e205741" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46715 } } }
I20260928 14:47:06.612951 30752 sys_catalog.cc:458] T 00000000000000000000000000000000 P e700f543e3034849a0487b6c0e205741 [sys.catalog]: This master's current role is: LEADER
I20260928 14:47:06.613185 30753 sys_catalog.cc:458] T 00000000000000000000000000000000 P e700f543e3034849a0487b6c0e205741 [sys.catalog]: This master's current role is: LEADER
I20260928 14:47:06.613828 30761 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:47:06.614346 30761 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:47:06.616066 30761 catalog_manager.cc:1347] Generated new cluster ID: 90384f73aedc4d02a6d5fff8bf197153
I20260928 14:47:06.616120 30761 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:47:06.617398 30751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:06.611869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.54:39199 (local address 127.0.0.1:46715)
0928 14:47:06.612096 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:06.612105 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:06.612167 (+ 62us) server_negotiation.cc:407] Connection header received
0928 14:47:06.612304 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:06.612311 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:06.612377 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:06.612520 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:06.612754 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:06.614263 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:06.615152 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:06.615830 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:06.615944 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:06.616724 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:06.616728 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:06.616728 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:47:06.616914 (+ 186us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:06.616948 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:47:06.616959 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:06.617056 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:06.617144 (+ 88us) server_negotiation.cc:299] Negotiation successful
0928 14:47:06.617173 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":112,"threads_started":1}
I20260928 14:47:06.620517 9349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:47:06.625072 30751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:06.621943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.54:43987 (local address 127.0.0.1:46715)
0928 14:47:06.621986 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:06.621987 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:06.622132 (+ 145us) server_negotiation.cc:407] Connection header received
0928 14:47:06.622201 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:06.622202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:06.622213 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:06.622263 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:06.622420 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:06.622925 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:06.623651 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:06.623951 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:06.624027 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:06.624790 (+ 763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:06.624792 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:06.624792 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:47:06.624797 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:06.624812 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:47:06.624816 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:06.624875 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:06.624948 (+ 73us) server_negotiation.cc:299] Negotiation successful
0928 14:47:06.624962 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 14:47:06.637925 30761 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:47:06.638736 30761 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:47:06.641992 30761 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e700f543e3034849a0487b6c0e205741: Generated new TSK 0
I20260928 14:47:06.642210 30761 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:47:06.644415 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 30679
W20260928 14:47:06.648555 30664 connection.cc:535] client connection to 127.0.0.1:46715 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46715): Connection reset by peer (error 104)
2026-09-28T14:47:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1340 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:47:06 dist-test-slave-pv0w krb5kdc[30778](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:47:06 dist-test-slave-pv0w krb5kdc[30778](info): set up 2 sockets
Sep 28 14:47:06 dist-test-slave-pv0w krb5kdc[30778](info): commencing operation
krb5kdc: starting...
W20260928 14:47:07.684791 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:47:07 dist-test-slave-pv0w krb5kdc[30778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:47:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:47:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260928 14:47:07.811554 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40201
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:47:07.896559 30791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:47:07.896811 30791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:47:07.896836 30791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:47:07.898108 30791 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:47:07.898168 30791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:47:07.898185 30791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:47:07.898197 30791 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:47:07.898211 30791 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:47:07.899948 30791 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40201
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606561545179-9349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30791
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:47:07.900246 30791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:47:07.900504 30791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:47:07.905789 30797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:47:07.905799 30796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:47:07.905963 30791 server_base.cc:1034] running on GCE node
W20260928 14:47:07.905786 30799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:47:07.906352 30791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:47:07.906680 30791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:47:07.907824 30791 hybrid_clock.cc:648] HybridClock initialized: now 1790606827907797 us; error 35 us; skew 500 ppm
Sep 28 14:47:07 dist-test-slave-pv0w krb5kdc[30778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606827, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:47:07.910754 30791 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260928 14:47:07.911329 30791 webserver.cc:460] Webserver started at http://127.0.0.1:41185/ using document root <none> and password file <none>
I20260928 14:47:07.911616 30791 fs_manager.cc:359] Metadata directory not provided
I20260928 14:47:07.911700 30791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:47:07.911896 30791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:47:07.913175 30791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "7b73f161275f4aabb98b8c263b8a0d16"
format_stamp: "Formatted at 2026-09-28 14:47:07 on dist-test-slave-pv0w"
server_key: "082f30cadca4b122bb1557f857e5feaf"
server_key_iv: "302f43f1cbf8c707c0a07c802215aea7"
server_key_version: "encryptionkey@0"
I20260928 14:47:07.913581 30791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "7b73f161275f4aabb98b8c263b8a0d16"
format_stamp: "Formatted at 2026-09-28 14:47:07 on dist-test-slave-pv0w"
server_key: "082f30cadca4b122bb1557f857e5feaf"
server_key_iv: "302f43f1cbf8c707c0a07c802215aea7"
server_key_version: "encryptionkey@0"
I20260928 14:47:07.915097 30791 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:47:07.916208 30806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:47:07.916587 30791 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:47:07.916723 30791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "7b73f161275f4aabb98b8c263b8a0d16"
format_stamp: "Formatted at 2026-09-28 14:47:07 on dist-test-slave-pv0w"
server_key: "082f30cadca4b122bb1557f857e5feaf"
server_key_iv: "302f43f1cbf8c707c0a07c802215aea7"
server_key_version: "encryptionkey@0"
I20260928 14:47:07.916813 30791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:47:07.920531 30791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:47:07.922683 30791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:47:07.922856 30791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:47:07.927182 30791 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40201
I20260928 14:47:07.927229 30858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40201 every 8 connection(s)
I20260928 14:47:07.927603 30791 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:47:07.928678 30859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:47:07.931069 30859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b73f161275f4aabb98b8c263b8a0d16: Bootstrap starting.
I20260928 14:47:07.931816 30859 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b73f161275f4aabb98b8c263b8a0d16: Neither blocks nor log segments found. Creating new log.
I20260928 14:47:07.932132 30859 log.cc:824] T 00000000000000000000000000000000 P 7b73f161275f4aabb98b8c263b8a0d16: Log is configured to *not* fsync() on all Append() calls
I20260928 14:47:07.932899 30859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b73f161275f4aabb98b8c263b8a0d16: No bootstrap required, opened a new log
I20260928 14:47:07.934643 30859 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b73f161275f4aabb98b8c263b8a0d16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b73f161275f4aabb98b8c263b8a0d16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40201 } }
I20260928 14:47:07.934795 30859 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b73f161275f4aabb98b8c263b8a0d16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:47:07.934818 30859 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b73f161275f4aabb98b8c263b8a0d16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b73f161275f4aabb98b8c263b8a0d16, State: Initialized, Role: FOLLOWER
I20260928 14:47:07.934995 30859 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b73f161275f4aabb98b8c263b8a0d16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b73f161275f4aabb98b8c263b8a0d16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40201 } }
I20260928 14:47:07.935081 30859 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b73f161275f4aabb98b8c263b8a0d16 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:47:07.935127 30859 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b73f161275f4aabb98b8c263b8a0d16 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:47:07.935204 30859 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b73f161275f4aabb98b8c263b8a0d16 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:47:07.935942 30859 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b73f161275f4aabb98b8c263b8a0d16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b73f161275f4aabb98b8c263b8a0d16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40201 } }
I20260928 14:47:07.936038 30859 leader_election.cc:302] T 00000000000000000000000000000000 P 7b73f161275f4aabb98b8c263b8a0d16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b73f161275f4aabb98b8c263b8a0d16; no voters:
I20260928 14:47:07.936261 30859 leader_election.cc:288] T 00000000000000000000000000000000 P 7b73f161275f4aabb98b8c263b8a0d16 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:47:07.936302 30862 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b73f161275f4aabb98b8c263b8a0d16 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:47:07.936554 30859 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b73f161275f4aabb98b8c263b8a0d16 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:47:07.936726 30862 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b73f161275f4aabb98b8c263b8a0d16 [term 1 LEADER]: Becoming Leader. State: Replica: 7b73f161275f4aabb98b8c263b8a0d16, State: Running, Role: LEADER
I20260928 14:47:07.936834 30862 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b73f161275f4aabb98b8c263b8a0d16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b73f161275f4aabb98b8c263b8a0d16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40201 } }
I20260928 14:47:07.937012 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 30791
I20260928 14:47:07.937136 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:47:07.937340 9349 external_mini_cluster.cc:1428] Setting key 22051ae0f68e9b08913f7dd27dcfd485
I20260928 14:47:07.937263 30863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b73f161275f4aabb98b8c263b8a0d16 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b73f161275f4aabb98b8c263b8a0d16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b73f161275f4aabb98b8c263b8a0d16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40201 } } }
I20260928 14:47:07.937263 30864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b73f161275f4aabb98b8c263b8a0d16 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b73f161275f4aabb98b8c263b8a0d16. Latest consensus state: current_term: 1 leader_uuid: "7b73f161275f4aabb98b8c263b8a0d16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b73f161275f4aabb98b8c263b8a0d16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40201 } } }
I20260928 14:47:07.937394 30863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b73f161275f4aabb98b8c263b8a0d16 [sys.catalog]: This master's current role is: LEADER
I20260928 14:47:07.937399 30864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b73f161275f4aabb98b8c263b8a0d16 [sys.catalog]: This master's current role is: LEADER
I20260928 14:47:07.937748 30866 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:47:07.938318 30866 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:47:07.940016 30870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:07.938099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.54:56625 (local address 127.0.0.1:40201)
0928 14:47:07.938276 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:07.938286 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:07.938323 (+ 37us) server_negotiation.cc:407] Connection header received
0928 14:47:07.938474 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:07.938482 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:07.938519 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:07.938702 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:07.939418 (+ 716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:07.939422 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:07.939422 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:47:07.939427 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:07.939472 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:47:07.939488 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:07.939699 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:07.939758 (+ 59us) server_negotiation.cc:299] Negotiation successful
0928 14:47:07.939804 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":77,"threads_started":1}
I20260928 14:47:07.940397 30866 catalog_manager.cc:1347] Generated new cluster ID: 6fba659327b74320844e4c98fc0d59cb
I20260928 14:47:07.940448 30866 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:47:07.941179 9349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:47:07.949262 30870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:07.942702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.54:55087 (local address 127.0.0.1:40201)
0928 14:47:07.942754 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:07.942756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:07.948020 (+ 5264us) server_negotiation.cc:407] Connection header received
0928 14:47:07.948114 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:07.948116 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:07.948122 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:07.948189 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:07.948986 (+ 797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:07.948987 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:07.948987 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:47:07.948991 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:07.949008 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:47:07.949013 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:07.949068 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:07.949159 (+ 91us) server_negotiation.cc:299] Negotiation successful
0928 14:47:07.949173 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260928 14:47:07.959205 30866 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:47:07.959422 30866 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:47:07.964213 30866 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b73f161275f4aabb98b8c263b8a0d16: Generated new TSK 0
I20260928 14:47:07.964427 30866 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:47:07.968411 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 30791
2026-09-28T14:47:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1324 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:47:08 dist-test-slave-pv0w krb5kdc[30890](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:47:08 dist-test-slave-pv0w krb5kdc[30890](info): set up 2 sockets
Sep 28 14:47:08 dist-test-slave-pv0w krb5kdc[30890](info): commencing operation
krb5kdc: starting...
W20260928 14:47:09.032770 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.046s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:47:09 dist-test-slave-pv0w krb5kdc[30890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:47:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:47:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.33.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
I20260928 14:47:09.165189 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:42123
--webserver_interface=127.9.33.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:44001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:42123
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:47:09.248566 30903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:47:09.248801 30903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:47:09.248827 30903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:47:09.250082 30903 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:47:09.250133 30903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:47:09.250146 30903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:47:09.250160 30903 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:47:09.250172 30903 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:47:09.251868 30903 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:44001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:42123
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:42123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30903
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:47:09.252161 30903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:47:09.252411 30903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:47:09.257476 30909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:47:09.257567 30911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:47:09.257727 30908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:47:09.257731 30903 server_base.cc:1034] running on GCE node
I20260928 14:47:09.258009 30903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:47:09.258287 30903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:47:09.259459 30903 hybrid_clock.cc:648] HybridClock initialized: now 1790606829259402 us; error 71 us; skew 500 ppm
Sep 28 14:47:09 dist-test-slave-pv0w krb5kdc[30890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606829, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:47:09.262288 30903 init.cc:377] Logged in from keytab as kudu/127.9.33.126@KRBTEST.COM (short username kudu)
I20260928 14:47:09.262854 30903 webserver.cc:460] Webserver started at http://127.9.33.126:41867/ using document root <none> and password file <none>
I20260928 14:47:09.263089 30903 fs_manager.cc:359] Metadata directory not provided
I20260928 14:47:09.263134 30903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:47:09.263303 30903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:47:09.264474 30903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "d9ab56b3b9c142da92968710a72e1593"
format_stamp: "Formatted at 2026-09-28 14:47:09 on dist-test-slave-pv0w"
server_key: "fbef414566867c466b85c8628b4d28a0"
server_key_iv: "75c468d189f1df548b46f58c70617486"
server_key_version: "encryptionkey@0"
I20260928 14:47:09.264833 30903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "d9ab56b3b9c142da92968710a72e1593"
format_stamp: "Formatted at 2026-09-28 14:47:09 on dist-test-slave-pv0w"
server_key: "fbef414566867c466b85c8628b4d28a0"
server_key_iv: "75c468d189f1df548b46f58c70617486"
server_key_version: "encryptionkey@0"
I20260928 14:47:09.266227 30903 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260928 14:47:09.267066 30918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:47:09.267287 30903 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:47:09.267400 30903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "d9ab56b3b9c142da92968710a72e1593"
format_stamp: "Formatted at 2026-09-28 14:47:09 on dist-test-slave-pv0w"
server_key: "fbef414566867c466b85c8628b4d28a0"
server_key_iv: "75c468d189f1df548b46f58c70617486"
server_key_version: "encryptionkey@0"
I20260928 14:47:09.267531 30903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:47:09.295886 30903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:47:09.298360 30903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:47:09.298547 30903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:47:09.302784 30903 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:42123
I20260928 14:47:09.302824 30970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:42123 every 8 connection(s)
I20260928 14:47:09.303194 30903 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:47:09.304229 30971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:47:09.306658 30971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9ab56b3b9c142da92968710a72e1593: Bootstrap starting.
I20260928 14:47:09.307379 30971 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9ab56b3b9c142da92968710a72e1593: Neither blocks nor log segments found. Creating new log.
I20260928 14:47:09.307687 30971 log.cc:824] T 00000000000000000000000000000000 P d9ab56b3b9c142da92968710a72e1593: Log is configured to *not* fsync() on all Append() calls
I20260928 14:47:09.308502 30971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9ab56b3b9c142da92968710a72e1593: No bootstrap required, opened a new log
I20260928 14:47:09.310134 30971 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9ab56b3b9c142da92968710a72e1593 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9ab56b3b9c142da92968710a72e1593" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 42123 } }
I20260928 14:47:09.310277 30971 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9ab56b3b9c142da92968710a72e1593 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:47:09.310300 30971 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9ab56b3b9c142da92968710a72e1593 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9ab56b3b9c142da92968710a72e1593, State: Initialized, Role: FOLLOWER
I20260928 14:47:09.310464 30971 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9ab56b3b9c142da92968710a72e1593 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9ab56b3b9c142da92968710a72e1593" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 42123 } }
I20260928 14:47:09.310530 30971 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9ab56b3b9c142da92968710a72e1593 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:47:09.310590 30971 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9ab56b3b9c142da92968710a72e1593 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:47:09.310648 30971 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9ab56b3b9c142da92968710a72e1593 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:47:09.310768 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 30903
I20260928 14:47:09.310879 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:47:09.311069 9349 external_mini_cluster.cc:1428] Setting key d1c56b6f4cac566c41afe248a167028a
I20260928 14:47:09.311873 30971 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9ab56b3b9c142da92968710a72e1593 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9ab56b3b9c142da92968710a72e1593" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 42123 } }
I20260928 14:47:09.312031 30971 leader_election.cc:302] T 00000000000000000000000000000000 P d9ab56b3b9c142da92968710a72e1593 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9ab56b3b9c142da92968710a72e1593; no voters:
I20260928 14:47:09.312229 30971 leader_election.cc:288] T 00000000000000000000000000000000 P d9ab56b3b9c142da92968710a72e1593 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:47:09.312249 30976 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9ab56b3b9c142da92968710a72e1593 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:47:09.312366 30976 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9ab56b3b9c142da92968710a72e1593 [term 1 LEADER]: Becoming Leader. State: Replica: d9ab56b3b9c142da92968710a72e1593, State: Running, Role: LEADER
I20260928 14:47:09.312521 30971 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9ab56b3b9c142da92968710a72e1593 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:47:09.312507 30976 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9ab56b3b9c142da92968710a72e1593 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9ab56b3b9c142da92968710a72e1593" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 42123 } }
I20260928 14:47:09.312989 30977 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9ab56b3b9c142da92968710a72e1593 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9ab56b3b9c142da92968710a72e1593. Latest consensus state: current_term: 1 leader_uuid: "d9ab56b3b9c142da92968710a72e1593" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9ab56b3b9c142da92968710a72e1593" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 42123 } } }
I20260928 14:47:09.313084 30977 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9ab56b3b9c142da92968710a72e1593 [sys.catalog]: This master's current role is: LEADER
I20260928 14:47:09.313323 30978 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9ab56b3b9c142da92968710a72e1593 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9ab56b3b9c142da92968710a72e1593" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9ab56b3b9c142da92968710a72e1593" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 42123 } } }
I20260928 14:47:09.313638 30978 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9ab56b3b9c142da92968710a72e1593 [sys.catalog]: This master's current role is: LEADER
I20260928 14:47:09.313833 30984 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:47:09.314386 30984 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:47:09.315972 30984 catalog_manager.cc:1347] Generated new cluster ID: 33964e3a663544e48745aa476f8d366b
I20260928 14:47:09.316025 30984 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 28 14:47:09 dist-test-slave-pv0w krb5kdc[30890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:47:09.324843 30975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:09.311642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46779 (local address 127.9.33.126:42123)
0928 14:47:09.311830 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:09.311840 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:09.311883 (+ 43us) server_negotiation.cc:407] Connection header received
0928 14:47:09.312161 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:09.312169 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:09.312233 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:09.313269 (+ 1036us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:09.313385 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:09.314882 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:09.316315 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:09.316970 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:09.317098 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:09.320101 (+ 3003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:09.320106 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:09.320106 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:47:09.320123 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:09.322752 (+ 2629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:09.323287 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:09.323292 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:09.323292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:09.323449 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:09.323745 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:09.323746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:09.323747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:09.324149 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:09.324356 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:09.324462 (+ 106us) server_negotiation.cc:299] Negotiation successful
0928 14:47:09.324592 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":104,"threads_started":1}
I20260928 14:47:09.325883 9349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:47:09.333086 30984 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:47:09.333729 30984 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:47:09.335378 30975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:09.327415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46293 (local address 127.9.33.126:42123)
0928 14:47:09.327458 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:09.327459 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:09.327672 (+ 213us) server_negotiation.cc:407] Connection header received
0928 14:47:09.327760 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:09.327761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:09.327773 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:09.327838 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:09.328470 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:09.329295 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:09.330305 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:09.330658 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:09.330735 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:09.332275 (+ 1540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:09.332277 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:09.332277 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:47:09.332286 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:09.334066 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:09.334520 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:09.334522 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:09.334522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:09.334587 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:09.334817 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:09.334818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:09.334818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:09.334961 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:09.335073 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:09.335215 (+ 142us) server_negotiation.cc:299] Negotiation successful
0928 14:47:09.335281 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 14:47:09.343662 30984 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9ab56b3b9c142da92968710a72e1593: Generated new TSK 0
I20260928 14:47:09.343926 30984 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:47:09.346366 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 30903
W20260928 14:47:09.350600 30888 connection.cc:535] client connection to 127.9.33.126:42123 recv error: Network error: failed to read from TLS socket (remote: 127.9.33.126:42123): Connection reset by peer (error 104)
2026-09-28T14:47:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1377 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:47:09 dist-test-slave-pv0w krb5kdc[31002](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:47:09 dist-test-slave-pv0w krb5kdc[31002](info): set up 2 sockets
Sep 28 14:47:09 dist-test-slave-pv0w krb5kdc[31002](info): commencing operation
krb5kdc: starting...
W20260928 14:47:10.404628 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:47:10 dist-test-slave-pv0w krb5kdc[31002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:47:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:47:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.33.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
I20260928 14:47:10.534844 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:40469
--webserver_interface=127.9.33.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:32957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:40469
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:47:10.621129 31015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:47:10.621410 31015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:47:10.621452 31015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:47:10.622732 31015 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:47:10.622813 31015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:47:10.622838 31015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:47:10.622859 31015 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:47:10.622890 31015 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:47:10.624583 31015 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:32957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:40469
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:40469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:47:10.624902 31015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:47:10.625161 31015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:47:10.630504 31021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:47:10.630496 31023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:47:10.630496 31020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:47:10.630879 31015 server_base.cc:1034] running on GCE node
I20260928 14:47:10.631165 31015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:47:10.631505 31015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:47:10.632674 31015 hybrid_clock.cc:648] HybridClock initialized: now 1790606830632639 us; error 50 us; skew 500 ppm
Sep 28 14:47:10 dist-test-slave-pv0w krb5kdc[31002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606830, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:47:10.640257 31015 init.cc:377] Logged in from keytab as kudu/127.9.33.126@KRBTEST.COM (short username kudu)
I20260928 14:47:10.640793 31015 webserver.cc:460] Webserver started at http://127.9.33.126:44159/ using document root <none> and password file <none>
I20260928 14:47:10.641045 31015 fs_manager.cc:359] Metadata directory not provided
I20260928 14:47:10.641120 31015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:47:10.641278 31015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:47:10.642467 31015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "66da6c40314b4a5c833e87f394a4752b"
format_stamp: "Formatted at 2026-09-28 14:47:10 on dist-test-slave-pv0w"
server_key: "b151dd021b685a1ec845085ecf3c3c91"
server_key_iv: "424cc7e2ff547b860e122cac55882e62"
server_key_version: "encryptionkey@0"
I20260928 14:47:10.642944 31015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "66da6c40314b4a5c833e87f394a4752b"
format_stamp: "Formatted at 2026-09-28 14:47:10 on dist-test-slave-pv0w"
server_key: "b151dd021b685a1ec845085ecf3c3c91"
server_key_iv: "424cc7e2ff547b860e122cac55882e62"
server_key_version: "encryptionkey@0"
I20260928 14:47:10.644567 31015 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:47:10.645493 31030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:47:10.645711 31015 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:47:10.645829 31015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "66da6c40314b4a5c833e87f394a4752b"
format_stamp: "Formatted at 2026-09-28 14:47:10 on dist-test-slave-pv0w"
server_key: "b151dd021b685a1ec845085ecf3c3c91"
server_key_iv: "424cc7e2ff547b860e122cac55882e62"
server_key_version: "encryptionkey@0"
I20260928 14:47:10.645921 31015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:47:10.664731 31015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:47:10.667037 31015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:47:10.667230 31015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:47:10.671463 31015 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:40469
I20260928 14:47:10.671500 31082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:40469 every 8 connection(s)
I20260928 14:47:10.671983 31015 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:47:10.672919 31083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:47:10.675333 31083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66da6c40314b4a5c833e87f394a4752b: Bootstrap starting.
I20260928 14:47:10.676049 31083 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66da6c40314b4a5c833e87f394a4752b: Neither blocks nor log segments found. Creating new log.
I20260928 14:47:10.676294 31083 log.cc:824] T 00000000000000000000000000000000 P 66da6c40314b4a5c833e87f394a4752b: Log is configured to *not* fsync() on all Append() calls
I20260928 14:47:10.677037 31083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66da6c40314b4a5c833e87f394a4752b: No bootstrap required, opened a new log
I20260928 14:47:10.678714 31083 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66da6c40314b4a5c833e87f394a4752b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66da6c40314b4a5c833e87f394a4752b" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40469 } }
I20260928 14:47:10.678876 31083 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66da6c40314b4a5c833e87f394a4752b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:47:10.678900 31083 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66da6c40314b4a5c833e87f394a4752b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66da6c40314b4a5c833e87f394a4752b, State: Initialized, Role: FOLLOWER
I20260928 14:47:10.679056 31083 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66da6c40314b4a5c833e87f394a4752b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66da6c40314b4a5c833e87f394a4752b" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40469 } }
I20260928 14:47:10.679143 31083 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66da6c40314b4a5c833e87f394a4752b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:47:10.679168 31083 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66da6c40314b4a5c833e87f394a4752b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:47:10.679198 31083 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66da6c40314b4a5c833e87f394a4752b [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:47:10.680102 31083 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66da6c40314b4a5c833e87f394a4752b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66da6c40314b4a5c833e87f394a4752b" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40469 } }
I20260928 14:47:10.680202 31083 leader_election.cc:302] T 00000000000000000000000000000000 P 66da6c40314b4a5c833e87f394a4752b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66da6c40314b4a5c833e87f394a4752b; no voters:
I20260928 14:47:10.680335 31083 leader_election.cc:288] T 00000000000000000000000000000000 P 66da6c40314b4a5c833e87f394a4752b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:47:10.680408 31086 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66da6c40314b4a5c833e87f394a4752b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:47:10.680611 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 31015
I20260928 14:47:10.680630 31086 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66da6c40314b4a5c833e87f394a4752b [term 1 LEADER]: Becoming Leader. State: Replica: 66da6c40314b4a5c833e87f394a4752b, State: Running, Role: LEADER
I20260928 14:47:10.680673 31083 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66da6c40314b4a5c833e87f394a4752b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:47:10.680732 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:47:10.680716 31086 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66da6c40314b4a5c833e87f394a4752b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66da6c40314b4a5c833e87f394a4752b" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40469 } }
I20260928 14:47:10.680943 9349 external_mini_cluster.cc:1428] Setting key 9b7bf72831427034e26f2274e51616bb
I20260928 14:47:10.681036 31088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66da6c40314b4a5c833e87f394a4752b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66da6c40314b4a5c833e87f394a4752b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66da6c40314b4a5c833e87f394a4752b" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40469 } } }
I20260928 14:47:10.681027 31086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66da6c40314b4a5c833e87f394a4752b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66da6c40314b4a5c833e87f394a4752b. Latest consensus state: current_term: 1 leader_uuid: "66da6c40314b4a5c833e87f394a4752b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66da6c40314b4a5c833e87f394a4752b" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40469 } } }
I20260928 14:47:10.681174 31086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66da6c40314b4a5c833e87f394a4752b [sys.catalog]: This master's current role is: LEADER
I20260928 14:47:10.681160 31088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66da6c40314b4a5c833e87f394a4752b [sys.catalog]: This master's current role is: LEADER
I20260928 14:47:10.681464 31093 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:47:10.682269 31093 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:47:10.684379 31093 catalog_manager.cc:1347] Generated new cluster ID: dabbaec9b93c4137b8ff2cfdfc8bd46a
I20260928 14:47:10.684475 31093 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:47:10.687626 31097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:10.681501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44167 (local address 127.9.33.126:40469)
0928 14:47:10.682169 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:10.682180 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:10.682264 (+ 84us) server_negotiation.cc:407] Connection header received
0928 14:47:10.682406 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:10.682414 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:10.682474 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:10.682629 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:10.682827 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:10.684086 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:10.685260 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:10.685900 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:10.686013 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:10.686862 (+ 849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:10.686866 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:10.686867 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:47:10.687086 (+ 219us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:10.687126 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:47:10.687141 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:10.687224 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:10.687355 (+ 131us) server_negotiation.cc:299] Negotiation successful
0928 14:47:10.687387 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":71,"threads_started":1}
I20260928 14:47:10.688756 9349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:47:10.694186 31097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:10.690435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48131 (local address 127.9.33.126:40469)
0928 14:47:10.690518 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:10.690520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:10.690581 (+ 61us) server_negotiation.cc:407] Connection header received
0928 14:47:10.690652 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:10.690652 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:10.690663 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:10.690716 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:10.690882 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:10.691624 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:10.692707 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:10.693043 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:10.693111 (+ 68us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:10.693901 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:10.693903 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:10.693903 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:47:10.693909 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:10.693927 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:47:10.693932 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:10.693988 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:10.694080 (+ 92us) server_negotiation.cc:299] Negotiation successful
0928 14:47:10.694094 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260928 14:47:10.704509 31093 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:47:10.705080 31093 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:47:10.707455 31093 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66da6c40314b4a5c833e87f394a4752b: Generated new TSK 0
I20260928 14:47:10.707669 31093 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:47:10.716387 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 31015
W20260928 14:47:10.720649 31000 connection.cc:535] client connection to 127.9.33.126:40469 recv error: Network error: failed to read from TLS socket (remote: 127.9.33.126:40469): Connection reset by peer (error 104)
2026-09-28T14:47:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1369 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:47:10 dist-test-slave-pv0w krb5kdc[31114](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:47:10 dist-test-slave-pv0w krb5kdc[31114](info): set up 2 sockets
Sep 28 14:47:10 dist-test-slave-pv0w krb5kdc[31114](info): commencing operation
krb5kdc: starting...
W20260928 14:47:11.757242 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:47:11 dist-test-slave-pv0w krb5kdc[31114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:47:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:47:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.33.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
I20260928 14:47:11.882421 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:42987
--webserver_interface=127.9.33.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:44791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:42987
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:47:11.964531 31127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:47:11.964782 31127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:47:11.964821 31127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:47:11.965978 31127 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:47:11.966050 31127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:47:11.966074 31127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:47:11.966104 31127 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:47:11.966126 31127 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:47:11.967643 31127 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:44791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:42987
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:42987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:47:11.967954 31127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:47:11.968189 31127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:47:11.973356 31135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:47:11.973348 31132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:47:11.973361 31133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:47:11.973677 31127 server_base.cc:1034] running on GCE node
I20260928 14:47:11.973950 31127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:47:11.974251 31127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:47:11.975402 31127 hybrid_clock.cc:648] HybridClock initialized: now 1790606831975365 us; error 53 us; skew 500 ppm
Sep 28 14:47:11 dist-test-slave-pv0w krb5kdc[31114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606831, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:47:11.977991 31127 init.cc:377] Logged in from keytab as kudu/127.9.33.126@KRBTEST.COM (short username kudu)
I20260928 14:47:11.978477 31127 webserver.cc:460] Webserver started at http://127.9.33.126:45439/ using document root <none> and password file <none>
I20260928 14:47:11.978729 31127 fs_manager.cc:359] Metadata directory not provided
I20260928 14:47:11.978796 31127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:47:11.978940 31127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:47:11.980088 31127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "f331779a21d94c03ab0ce27600308266"
format_stamp: "Formatted at 2026-09-28 14:47:11 on dist-test-slave-pv0w"
server_key: "99bbb804ad00049ef76993fb49774623"
server_key_iv: "c84d295157b14924f671d63376698a2e"
server_key_version: "encryptionkey@0"
I20260928 14:47:11.980456 31127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "f331779a21d94c03ab0ce27600308266"
format_stamp: "Formatted at 2026-09-28 14:47:11 on dist-test-slave-pv0w"
server_key: "99bbb804ad00049ef76993fb49774623"
server_key_iv: "c84d295157b14924f671d63376698a2e"
server_key_version: "encryptionkey@0"
I20260928 14:47:11.981873 31127 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:47:11.982612 31142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:47:11.982841 31127 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:47:11.982955 31127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "f331779a21d94c03ab0ce27600308266"
format_stamp: "Formatted at 2026-09-28 14:47:11 on dist-test-slave-pv0w"
server_key: "99bbb804ad00049ef76993fb49774623"
server_key_iv: "c84d295157b14924f671d63376698a2e"
server_key_version: "encryptionkey@0"
I20260928 14:47:11.983038 31127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:47:11.986605 31127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:47:11.988802 31127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:47:11.988958 31127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:47:11.993425 31127 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:42987
I20260928 14:47:11.993456 31194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:42987 every 8 connection(s)
I20260928 14:47:11.993816 31127 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:47:11.994814 31195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:47:11.997191 31195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f331779a21d94c03ab0ce27600308266: Bootstrap starting.
I20260928 14:47:11.997221 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 31127
I20260928 14:47:11.997326 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:47:11.997534 9349 external_mini_cluster.cc:1428] Setting key b391922e872a2eb4dd43b9d1635d6c09
I20260928 14:47:11.997967 31195 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f331779a21d94c03ab0ce27600308266: Neither blocks nor log segments found. Creating new log.
I20260928 14:47:11.998338 31195 log.cc:824] T 00000000000000000000000000000000 P f331779a21d94c03ab0ce27600308266: Log is configured to *not* fsync() on all Append() calls
I20260928 14:47:11.999105 31195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f331779a21d94c03ab0ce27600308266: No bootstrap required, opened a new log
I20260928 14:47:12.000289 31198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:11.998057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43929 (local address 127.9.33.126:42987)
0928 14:47:11.998273 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:11.998282 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:11.998324 (+ 42us) server_negotiation.cc:407] Connection header received
0928 14:47:11.998932 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:11.998941 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:11.998977 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:11.999142 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:11.999843 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:11.999847 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:11.999847 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:47:11.999853 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:11.999899 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:47:11.999915 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:12.000001 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:12.000039 (+ 38us) server_negotiation.cc:299] Negotiation successful
0928 14:47:12.000079 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":89,"threads_started":1}
I20260928 14:47:12.001386 9349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:47:12.001718 31195 raft_consensus.cc:348] T 00000000000000000000000000000000 P f331779a21d94c03ab0ce27600308266 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f331779a21d94c03ab0ce27600308266" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 42987 } }
I20260928 14:47:12.001901 31195 raft_consensus.cc:374] T 00000000000000000000000000000000 P f331779a21d94c03ab0ce27600308266 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:47:12.001937 31195 raft_consensus.cc:729] T 00000000000000000000000000000000 P f331779a21d94c03ab0ce27600308266 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f331779a21d94c03ab0ce27600308266, State: Initialized, Role: FOLLOWER
I20260928 14:47:12.002066 31195 consensus_queue.cc:260] T 00000000000000000000000000000000 P f331779a21d94c03ab0ce27600308266 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f331779a21d94c03ab0ce27600308266" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 42987 } }
I20260928 14:47:12.002159 31195 raft_consensus.cc:388] T 00000000000000000000000000000000 P f331779a21d94c03ab0ce27600308266 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:47:12.002197 31195 raft_consensus.cc:482] T 00000000000000000000000000000000 P f331779a21d94c03ab0ce27600308266 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:47:12.002251 31195 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f331779a21d94c03ab0ce27600308266 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:47:12.003042 31195 raft_consensus.cc:504] T 00000000000000000000000000000000 P f331779a21d94c03ab0ce27600308266 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f331779a21d94c03ab0ce27600308266" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 42987 } }
I20260928 14:47:12.003166 31195 leader_election.cc:302] T 00000000000000000000000000000000 P f331779a21d94c03ab0ce27600308266 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f331779a21d94c03ab0ce27600308266; no voters:
I20260928 14:47:12.003337 31195 leader_election.cc:288] T 00000000000000000000000000000000 P f331779a21d94c03ab0ce27600308266 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:47:12.003361 31205 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f331779a21d94c03ab0ce27600308266 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:47:12.003517 31205 raft_consensus.cc:686] T 00000000000000000000000000000000 P f331779a21d94c03ab0ce27600308266 [term 1 LEADER]: Becoming Leader. State: Replica: f331779a21d94c03ab0ce27600308266, State: Running, Role: LEADER
I20260928 14:47:12.003659 31195 sys_catalog.cc:564] T 00000000000000000000000000000000 P f331779a21d94c03ab0ce27600308266 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:47:12.003674 31205 consensus_queue.cc:237] T 00000000000000000000000000000000 P f331779a21d94c03ab0ce27600308266 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f331779a21d94c03ab0ce27600308266" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 42987 } }
I20260928 14:47:12.004172 31207 sys_catalog.cc:455] T 00000000000000000000000000000000 P f331779a21d94c03ab0ce27600308266 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f331779a21d94c03ab0ce27600308266. Latest consensus state: current_term: 1 leader_uuid: "f331779a21d94c03ab0ce27600308266" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f331779a21d94c03ab0ce27600308266" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 42987 } } }
I20260928 14:47:12.004267 31198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:12.003013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49915 (local address 127.9.33.126:42987)
0928 14:47:12.003076 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:12.003078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:12.003188 (+ 110us) server_negotiation.cc:407] Connection header received
0928 14:47:12.003284 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:12.003285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:12.003291 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:12.003347 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:12.003996 (+ 649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:12.003997 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:12.003998 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:47:12.004002 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:12.004017 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:47:12.004022 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:12.004063 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:12.004121 (+ 58us) server_negotiation.cc:299] Negotiation successful
0928 14:47:12.004136 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260928 14:47:12.004449 31206 sys_catalog.cc:455] T 00000000000000000000000000000000 P f331779a21d94c03ab0ce27600308266 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f331779a21d94c03ab0ce27600308266" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f331779a21d94c03ab0ce27600308266" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 42987 } } }
I20260928 14:47:12.004520 31206 sys_catalog.cc:458] T 00000000000000000000000000000000 P f331779a21d94c03ab0ce27600308266 [sys.catalog]: This master's current role is: LEADER
I20260928 14:47:12.004287 31207 sys_catalog.cc:458] T 00000000000000000000000000000000 P f331779a21d94c03ab0ce27600308266 [sys.catalog]: This master's current role is: LEADER
W20260928 14:47:12.006054 31221 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f331779a21d94c03ab0ce27600308266: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:47:12.006129 31221 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:47:12.006181 31211 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:47:12.006330 31211 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:47:12.007804 31211 catalog_manager.cc:1347] Generated new cluster ID: 80cf43b855cb474485a0c8007cf9e324
I20260928 14:47:12.007854 31211 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:47:12.015040 31211 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:47:12.015213 31211 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:47:12.019515 31211 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f331779a21d94c03ab0ce27600308266: Generated new TSK 0
I20260928 14:47:12.019713 31211 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:47:12.022944 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 31127
2026-09-28T14:47:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1306 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:47:12 dist-test-slave-pv0w krb5kdc[31226](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:47:12 dist-test-slave-pv0w krb5kdc[31226](info): set up 2 sockets
Sep 28 14:47:12 dist-test-slave-pv0w krb5kdc[31226](info): commencing operation
krb5kdc: starting...
W20260928 14:47:13.052556 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:47:13 dist-test-slave-pv0w krb5kdc[31226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606833, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:47:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:47:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.33.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
I20260928 14:47:13.179342 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:34131
--webserver_interface=127.9.33.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:33029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:34131
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:47:13.261420 31239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:47:13.261678 31239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:47:13.261715 31239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:47:13.262910 31239 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:47:13.262981 31239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:47:13.263006 31239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:47:13.263028 31239 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:47:13.263056 31239 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:47:13.264605 31239 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:33029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:34131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:34131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:47:13.264904 31239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:47:13.265143 31239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:47:13.270133 31245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:47:13.270139 31247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:47:13.270187 31244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:47:13.270156 31239 server_base.cc:1034] running on GCE node
I20260928 14:47:13.270628 31239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:47:13.270891 31239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:47:13.272043 31239 hybrid_clock.cc:648] HybridClock initialized: now 1790606833272016 us; error 49 us; skew 500 ppm
Sep 28 14:47:13 dist-test-slave-pv0w krb5kdc[31226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606833, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:47:13.274497 31239 init.cc:377] Logged in from keytab as kudu/127.9.33.126@KRBTEST.COM (short username kudu)
I20260928 14:47:13.274885 31239 webserver.cc:460] Webserver started at http://127.9.33.126:34369/ using document root <none> and password file <none>
I20260928 14:47:13.275076 31239 fs_manager.cc:359] Metadata directory not provided
I20260928 14:47:13.275125 31239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:47:13.275211 31239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:47:13.276361 31239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "39295df51ee84f51ad163d84ff9b2a3b"
format_stamp: "Formatted at 2026-09-28 14:47:13 on dist-test-slave-pv0w"
server_key: "c5efd445c6421131101c48c9e7394af9"
server_key_iv: "4f996ddfc4b28725aae6a24c505af3c2"
server_key_version: "encryptionkey@0"
I20260928 14:47:13.276665 31239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "39295df51ee84f51ad163d84ff9b2a3b"
format_stamp: "Formatted at 2026-09-28 14:47:13 on dist-test-slave-pv0w"
server_key: "c5efd445c6421131101c48c9e7394af9"
server_key_iv: "4f996ddfc4b28725aae6a24c505af3c2"
server_key_version: "encryptionkey@0"
I20260928 14:47:13.277944 31239 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:47:13.278677 31254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:47:13.278851 31239 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:47:13.278959 31239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "39295df51ee84f51ad163d84ff9b2a3b"
format_stamp: "Formatted at 2026-09-28 14:47:13 on dist-test-slave-pv0w"
server_key: "c5efd445c6421131101c48c9e7394af9"
server_key_iv: "4f996ddfc4b28725aae6a24c505af3c2"
server_key_version: "encryptionkey@0"
I20260928 14:47:13.279034 31239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:47:13.305747 31239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:47:13.307688 31239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:47:13.307888 31239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:47:13.311818 31239 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:34131
I20260928 14:47:13.311887 31306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:34131 every 8 connection(s)
I20260928 14:47:13.312258 31239 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:47:13.313150 31307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:47:13.314416 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 31239
I20260928 14:47:13.314563 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:47:13.314787 9349 external_mini_cluster.cc:1428] Setting key efc5fe6fec683b1b3a3662e3cd1360d3
I20260928 14:47:13.316077 31307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39295df51ee84f51ad163d84ff9b2a3b: Bootstrap starting.
I20260928 14:47:13.316778 31307 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39295df51ee84f51ad163d84ff9b2a3b: Neither blocks nor log segments found. Creating new log.
I20260928 14:47:13.317092 31307 log.cc:824] T 00000000000000000000000000000000 P 39295df51ee84f51ad163d84ff9b2a3b: Log is configured to *not* fsync() on all Append() calls
I20260928 14:47:13.317878 31307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39295df51ee84f51ad163d84ff9b2a3b: No bootstrap required, opened a new log
I20260928 14:47:13.320302 31307 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39295df51ee84f51ad163d84ff9b2a3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39295df51ee84f51ad163d84ff9b2a3b" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 34131 } }
I20260928 14:47:13.320508 31307 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39295df51ee84f51ad163d84ff9b2a3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:47:13.320564 31307 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39295df51ee84f51ad163d84ff9b2a3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39295df51ee84f51ad163d84ff9b2a3b, State: Initialized, Role: FOLLOWER
I20260928 14:47:13.320704 31307 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39295df51ee84f51ad163d84ff9b2a3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39295df51ee84f51ad163d84ff9b2a3b" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 34131 } }
I20260928 14:47:13.320794 31307 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39295df51ee84f51ad163d84ff9b2a3b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:47:13.320837 31307 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39295df51ee84f51ad163d84ff9b2a3b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:47:13.320897 31307 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39295df51ee84f51ad163d84ff9b2a3b [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:47:13.321462 31307 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39295df51ee84f51ad163d84ff9b2a3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39295df51ee84f51ad163d84ff9b2a3b" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 34131 } }
I20260928 14:47:13.321551 31307 leader_election.cc:302] T 00000000000000000000000000000000 P 39295df51ee84f51ad163d84ff9b2a3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39295df51ee84f51ad163d84ff9b2a3b; no voters:
I20260928 14:47:13.321761 31307 leader_election.cc:288] T 00000000000000000000000000000000 P 39295df51ee84f51ad163d84ff9b2a3b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:47:13.321787 31312 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39295df51ee84f51ad163d84ff9b2a3b [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:47:13.321898 31312 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39295df51ee84f51ad163d84ff9b2a3b [term 1 LEADER]: Becoming Leader. State: Replica: 39295df51ee84f51ad163d84ff9b2a3b, State: Running, Role: LEADER
I20260928 14:47:13.322036 31307 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39295df51ee84f51ad163d84ff9b2a3b [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:47:13.322042 31312 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39295df51ee84f51ad163d84ff9b2a3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39295df51ee84f51ad163d84ff9b2a3b" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 34131 } }
Sep 28 14:47:13 dist-test-slave-pv0w krb5kdc[31226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606833, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:47:13.322475 31312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39295df51ee84f51ad163d84ff9b2a3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39295df51ee84f51ad163d84ff9b2a3b. Latest consensus state: current_term: 1 leader_uuid: "39295df51ee84f51ad163d84ff9b2a3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39295df51ee84f51ad163d84ff9b2a3b" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 34131 } } }
I20260928 14:47:13.322571 31312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39295df51ee84f51ad163d84ff9b2a3b [sys.catalog]: This master's current role is: LEADER
I20260928 14:47:13.322492 31313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39295df51ee84f51ad163d84ff9b2a3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39295df51ee84f51ad163d84ff9b2a3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39295df51ee84f51ad163d84ff9b2a3b" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 34131 } } }
I20260928 14:47:13.322860 31313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39295df51ee84f51ad163d84ff9b2a3b [sys.catalog]: This master's current role is: LEADER
I20260928 14:47:13.323773 31317 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:47:13.324721 31317 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:47:13.326422 31317 catalog_manager.cc:1347] Generated new cluster ID: 9f80305864674f9987da17ac012a6a17
I20260928 14:47:13.326473 31317 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:47:13.327157 31310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:13.315389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35395 (local address 127.9.33.126:34131)
0928 14:47:13.315620 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:13.315632 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:13.315670 (+ 38us) server_negotiation.cc:407] Connection header received
0928 14:47:13.316258 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:13.316268 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:13.316329 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:13.316498 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:13.317183 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:13.318593 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:13.319404 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:13.320160 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:13.320275 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:13.323113 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:13.323117 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:13.323117 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:47:13.323129 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:13.325232 (+ 2103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:13.325655 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:13.325658 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:13.325659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:13.325778 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:13.326193 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:13.326194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:13.326194 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:13.326526 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:13.326700 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:13.326816 (+ 116us) server_negotiation.cc:299] Negotiation successful
0928 14:47:13.326954 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":110,"threads_started":1}
I20260928 14:47:13.328187 9349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:47:13.343349 31310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:13.329687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56929 (local address 127.9.33.126:34131)
0928 14:47:13.329726 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:13.329728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:13.335986 (+ 6258us) server_negotiation.cc:407] Connection header received
0928 14:47:13.336049 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:13.336051 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:13.336064 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:13.336139 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:13.336817 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:13.337608 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:13.338602 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:13.338955 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:13.339028 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:13.340487 (+ 1459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:13.340489 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:13.340489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:47:13.340497 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:13.342149 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:13.342545 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:13.342547 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:13.342547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:13.342612 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:13.342809 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:13.342810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:13.342810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:13.342951 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:13.343053 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:13.343162 (+ 109us) server_negotiation.cc:299] Negotiation successful
0928 14:47:13.343244 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 14:47:13.346860 31317 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:47:13.347616 31317 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:47:13.355418 31317 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39295df51ee84f51ad163d84ff9b2a3b: Generated new TSK 0
I20260928 14:47:13.355670 31317 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:47:13.358428 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 31239
2026-09-28T14:47:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1335 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:47:13 dist-test-slave-pv0w krb5kdc[31338](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:47:13 dist-test-slave-pv0w krb5kdc[31338](info): set up 2 sockets
Sep 28 14:47:13 dist-test-slave-pv0w krb5kdc[31338](info): commencing operation
krb5kdc: starting...
W20260928 14:47:14.392652 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:47:14 dist-test-slave-pv0w krb5kdc[31338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606834, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:47:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:47:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.33.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
I20260928 14:47:14.519194 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:46761
--webserver_interface=127.9.33.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:44483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:46761
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:47:14.601351 31351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:47:14.601572 31351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:47:14.601598 31351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:47:14.602727 31351 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:47:14.602766 31351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:47:14.602779 31351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:47:14.602792 31351 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:47:14.602804 31351 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:47:14.604271 31351 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:44483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:46761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:46761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:47:14.604473 31351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:47:14.604646 31351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:47:14.609876 31359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:47:14.609876 31356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:47:14.609862 31357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:47:14.609908 31351 server_base.cc:1034] running on GCE node
I20260928 14:47:14.610343 31351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:47:14.610648 31351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:47:14.611797 31351 hybrid_clock.cc:648] HybridClock initialized: now 1790606834611787 us; error 43 us; skew 500 ppm
Sep 28 14:47:14 dist-test-slave-pv0w krb5kdc[31338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606834, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:47:14.614392 31351 init.cc:377] Logged in from keytab as kudu/127.9.33.126@KRBTEST.COM (short username kudu)
I20260928 14:47:14.614864 31351 webserver.cc:460] Webserver started at http://127.9.33.126:34695/ using document root <none> and password file <none>
I20260928 14:47:14.615096 31351 fs_manager.cc:359] Metadata directory not provided
I20260928 14:47:14.615165 31351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:47:14.615324 31351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:47:14.616508 31351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "a61f271da738411ab494725b88bf5736"
format_stamp: "Formatted at 2026-09-28 14:47:14 on dist-test-slave-pv0w"
server_key: "d2df92f2bd1b1076c2681ed92a7a78b3"
server_key_iv: "b6422f3d5f32434bc754d74973482112"
server_key_version: "encryptionkey@0"
I20260928 14:47:14.616900 31351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "a61f271da738411ab494725b88bf5736"
format_stamp: "Formatted at 2026-09-28 14:47:14 on dist-test-slave-pv0w"
server_key: "d2df92f2bd1b1076c2681ed92a7a78b3"
server_key_iv: "b6422f3d5f32434bc754d74973482112"
server_key_version: "encryptionkey@0"
I20260928 14:47:14.618470 31351 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:47:14.619314 31366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:47:14.619520 31351 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:47:14.619647 31351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "a61f271da738411ab494725b88bf5736"
format_stamp: "Formatted at 2026-09-28 14:47:14 on dist-test-slave-pv0w"
server_key: "d2df92f2bd1b1076c2681ed92a7a78b3"
server_key_iv: "b6422f3d5f32434bc754d74973482112"
server_key_version: "encryptionkey@0"
I20260928 14:47:14.619756 31351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:47:14.630661 31351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:47:14.633211 31351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:47:14.633390 31351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:47:14.637594 31351 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:46761
I20260928 14:47:14.637636 31418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:46761 every 8 connection(s)
I20260928 14:47:14.638010 31351 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:47:14.638899 31419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:47:14.641315 31419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a61f271da738411ab494725b88bf5736: Bootstrap starting.
I20260928 14:47:14.641933 31419 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a61f271da738411ab494725b88bf5736: Neither blocks nor log segments found. Creating new log.
I20260928 14:47:14.642233 31419 log.cc:824] T 00000000000000000000000000000000 P a61f271da738411ab494725b88bf5736: Log is configured to *not* fsync() on all Append() calls
I20260928 14:47:14.642979 31419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a61f271da738411ab494725b88bf5736: No bootstrap required, opened a new log
I20260928 14:47:14.644416 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 31351
I20260928 14:47:14.644569 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:47:14.644773 9349 external_mini_cluster.cc:1428] Setting key f8f5b8d897313a5ce84234f300505299
I20260928 14:47:14.644836 31419 raft_consensus.cc:348] T 00000000000000000000000000000000 P a61f271da738411ab494725b88bf5736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a61f271da738411ab494725b88bf5736" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 46761 } }
I20260928 14:47:14.645071 31419 raft_consensus.cc:374] T 00000000000000000000000000000000 P a61f271da738411ab494725b88bf5736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:47:14.645155 31419 raft_consensus.cc:729] T 00000000000000000000000000000000 P a61f271da738411ab494725b88bf5736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a61f271da738411ab494725b88bf5736, State: Initialized, Role: FOLLOWER
I20260928 14:47:14.645325 31419 consensus_queue.cc:260] T 00000000000000000000000000000000 P a61f271da738411ab494725b88bf5736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a61f271da738411ab494725b88bf5736" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 46761 } }
I20260928 14:47:14.645529 31419 raft_consensus.cc:388] T 00000000000000000000000000000000 P a61f271da738411ab494725b88bf5736 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:47:14.645571 31419 raft_consensus.cc:482] T 00000000000000000000000000000000 P a61f271da738411ab494725b88bf5736 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:47:14.645627 31419 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a61f271da738411ab494725b88bf5736 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:47:14.646823 31419 raft_consensus.cc:504] T 00000000000000000000000000000000 P a61f271da738411ab494725b88bf5736 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a61f271da738411ab494725b88bf5736" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 46761 } }
I20260928 14:47:14.646936 31419 leader_election.cc:302] T 00000000000000000000000000000000 P a61f271da738411ab494725b88bf5736 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a61f271da738411ab494725b88bf5736; no voters:
I20260928 14:47:14.647085 31419 leader_election.cc:288] T 00000000000000000000000000000000 P a61f271da738411ab494725b88bf5736 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:47:14.647148 31424 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a61f271da738411ab494725b88bf5736 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:47:14.647317 31424 raft_consensus.cc:686] T 00000000000000000000000000000000 P a61f271da738411ab494725b88bf5736 [term 1 LEADER]: Becoming Leader. State: Replica: a61f271da738411ab494725b88bf5736, State: Running, Role: LEADER
I20260928 14:47:14.647332 31419 sys_catalog.cc:564] T 00000000000000000000000000000000 P a61f271da738411ab494725b88bf5736 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:47:14.647475 31424 consensus_queue.cc:237] T 00000000000000000000000000000000 P a61f271da738411ab494725b88bf5736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a61f271da738411ab494725b88bf5736" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 46761 } }
I20260928 14:47:14.648757 31426 sys_catalog.cc:455] T 00000000000000000000000000000000 P a61f271da738411ab494725b88bf5736 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a61f271da738411ab494725b88bf5736" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a61f271da738411ab494725b88bf5736" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 46761 } } }
I20260928 14:47:14.648860 31426 sys_catalog.cc:458] T 00000000000000000000000000000000 P a61f271da738411ab494725b88bf5736 [sys.catalog]: This master's current role is: LEADER
I20260928 14:47:14.648983 31425 sys_catalog.cc:455] T 00000000000000000000000000000000 P a61f271da738411ab494725b88bf5736 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a61f271da738411ab494725b88bf5736. Latest consensus state: current_term: 1 leader_uuid: "a61f271da738411ab494725b88bf5736" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a61f271da738411ab494725b88bf5736" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 46761 } } }
I20260928 14:47:14.649058 31425 sys_catalog.cc:458] T 00000000000000000000000000000000 P a61f271da738411ab494725b88bf5736 [sys.catalog]: This master's current role is: LEADER
W20260928 14:47:14.649461 31439 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a61f271da738411ab494725b88bf5736: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:47:14.649502 31439 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:47:14.649570 31440 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:47:14.649760 31440 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:47:14.651369 31440 catalog_manager.cc:1347] Generated new cluster ID: 5d238ef6639e461daf471d5aa1c9a898
I20260928 14:47:14.651386 31423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:14.645429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55159 (local address 127.9.33.126:46761)
0928 14:47:14.645706 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:14.645716 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:14.645748 (+ 32us) server_negotiation.cc:407] Connection header received
0928 14:47:14.645871 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:14.645878 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:14.645933 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:14.646088 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:14.646325 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:14.647745 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:14.648991 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:14.649615 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:14.649725 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:14.650607 (+ 882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:14.650612 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:14.650612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:47:14.650829 (+ 217us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:14.650871 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:47:14.650886 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:14.650958 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:14.651058 (+ 100us) server_negotiation.cc:299] Negotiation successful
0928 14:47:14.651129 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":126,"threads_started":1}
I20260928 14:47:14.651504 31440 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:47:14.652477 9349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:47:14.658733 31423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:14.653883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33611 (local address 127.9.33.126:46761)
0928 14:47:14.653920 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:14.653922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:14.655926 (+ 2004us) server_negotiation.cc:407] Connection header received
0928 14:47:14.656001 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:14.656002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:14.656013 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:14.656067 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:14.656226 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:14.656743 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:14.657469 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:14.657680 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:14.657768 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:14.658484 (+ 716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:14.658486 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:14.658486 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:47:14.658492 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:14.658506 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:47:14.658510 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:14.658557 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:14.658613 (+ 56us) server_negotiation.cc:299] Negotiation successful
0928 14:47:14.658639 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 14:47:14.681216 31440 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:47:14.681829 31440 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:47:14.685755 31440 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a61f271da738411ab494725b88bf5736: Generated new TSK 0
I20260928 14:47:14.685973 31440 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:47:14.696799 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 31351
2026-09-28T14:47:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1339 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:47:14 dist-test-slave-pv0w krb5kdc[31450](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:47:14 dist-test-slave-pv0w krb5kdc[31450](info): set up 2 sockets
Sep 28 14:47:14 dist-test-slave-pv0w krb5kdc[31450](info): commencing operation
krb5kdc: starting...
W20260928 14:47:15.740703 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:47:15 dist-test-slave-pv0w krb5kdc[31450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606835, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:47:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:47:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.33.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
I20260928 14:47:15.867167 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:34295
--webserver_interface=127.9.33.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:39527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:34295
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:47:15.951222 31463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:47:15.951504 31463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:47:15.951550 31463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:47:15.952821 31463 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:47:15.952911 31463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:47:15.952936 31463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:47:15.952958 31463 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:47:15.952989 31463 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:47:15.954628 31463 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:39527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:34295
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:34295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31463
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:47:15.954943 31463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:47:15.955192 31463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:47:15.960254 31469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:47:15.960263 31471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:47:15.960289 31468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:47:15.960541 31463 server_base.cc:1034] running on GCE node
I20260928 14:47:15.960798 31463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:47:15.961071 31463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:47:15.962218 31463 hybrid_clock.cc:648] HybridClock initialized: now 1790606835962202 us; error 32 us; skew 500 ppm
Sep 28 14:47:15 dist-test-slave-pv0w krb5kdc[31450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606835, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:47:15.971840 31463 init.cc:377] Logged in from keytab as kudu/127.9.33.126@KRBTEST.COM (short username kudu)
I20260928 14:47:15.972337 31463 webserver.cc:460] Webserver started at http://127.9.33.126:35955/ using document root <none> and password file <none>
I20260928 14:47:15.972548 31463 fs_manager.cc:359] Metadata directory not provided
I20260928 14:47:15.972591 31463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:47:15.972692 31463 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:47:15.973805 31463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "4d154bf2e9be473ab119c009a8ac51e2"
format_stamp: "Formatted at 2026-09-28 14:47:15 on dist-test-slave-pv0w"
server_key: "d97f296effcbdceb40b43ae3f1859fc8"
server_key_iv: "1a8bfce50868c7cfcf21ad7b39c861e4"
server_key_version: "encryptionkey@0"
I20260928 14:47:15.974123 31463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "4d154bf2e9be473ab119c009a8ac51e2"
format_stamp: "Formatted at 2026-09-28 14:47:15 on dist-test-slave-pv0w"
server_key: "d97f296effcbdceb40b43ae3f1859fc8"
server_key_iv: "1a8bfce50868c7cfcf21ad7b39c861e4"
server_key_version: "encryptionkey@0"
I20260928 14:47:15.975621 31463 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:47:15.976598 31478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:47:15.976799 31463 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:47:15.976891 31463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "4d154bf2e9be473ab119c009a8ac51e2"
format_stamp: "Formatted at 2026-09-28 14:47:15 on dist-test-slave-pv0w"
server_key: "d97f296effcbdceb40b43ae3f1859fc8"
server_key_iv: "1a8bfce50868c7cfcf21ad7b39c861e4"
server_key_version: "encryptionkey@0"
I20260928 14:47:15.976958 31463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:47:15.980479 31463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:47:15.982736 31463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:47:15.982856 31463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:47:15.986912 31463 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:34295
I20260928 14:47:15.986944 31530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:34295 every 8 connection(s)
I20260928 14:47:15.987303 31463 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:47:15.988430 31531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:47:15.990800 31531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d154bf2e9be473ab119c009a8ac51e2: Bootstrap starting.
I20260928 14:47:15.991415 31531 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d154bf2e9be473ab119c009a8ac51e2: Neither blocks nor log segments found. Creating new log.
I20260928 14:47:15.991714 31531 log.cc:824] T 00000000000000000000000000000000 P 4d154bf2e9be473ab119c009a8ac51e2: Log is configured to *not* fsync() on all Append() calls
I20260928 14:47:15.992615 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 31463
I20260928 14:47:15.992668 31531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d154bf2e9be473ab119c009a8ac51e2: No bootstrap required, opened a new log
I20260928 14:47:15.992746 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:47:15.992954 9349 external_mini_cluster.cc:1428] Setting key f3550344d5e1f6c16a9e10c9dbafb5e2
I20260928 14:47:15.994692 31531 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d154bf2e9be473ab119c009a8ac51e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d154bf2e9be473ab119c009a8ac51e2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 34295 } }
I20260928 14:47:15.994871 31531 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d154bf2e9be473ab119c009a8ac51e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260928 14:47:15.994938 31531 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d154bf2e9be473ab119c009a8ac51e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d154bf2e9be473ab119c009a8ac51e2, State: Initialized, Role: FOLLOWER
I20260928 14:47:15.995150 31531 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d154bf2e9be473ab119c009a8ac51e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d154bf2e9be473ab119c009a8ac51e2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 34295 } }
I20260928 14:47:15.995260 31531 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d154bf2e9be473ab119c009a8ac51e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260928 14:47:15.995301 31531 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d154bf2e9be473ab119c009a8ac51e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260928 14:47:15.995378 31531 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d154bf2e9be473ab119c009a8ac51e2 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:47:15.996122 31535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:15.993519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48715 (local address 127.9.33.126:34295)
0928 14:47:15.993745 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:15.993753 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:15.993780 (+ 27us) server_negotiation.cc:407] Connection header received
0928 14:47:15.994681 (+ 901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:15.994689 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:15.994720 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:15.994877 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:15.995632 (+ 755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:15.995635 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:15.995635 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:47:15.995639 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:15.995699 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:47:15.995712 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:15.995819 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:15.995898 (+ 79us) server_negotiation.cc:299] Negotiation successful
0928 14:47:15.995944 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":99,"threads_started":1}
I20260928 14:47:15.996688 31531 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d154bf2e9be473ab119c009a8ac51e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d154bf2e9be473ab119c009a8ac51e2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 34295 } }
I20260928 14:47:15.996827 31531 leader_election.cc:302] T 00000000000000000000000000000000 P 4d154bf2e9be473ab119c009a8ac51e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d154bf2e9be473ab119c009a8ac51e2; no voters:
I20260928 14:47:15.997089 31531 leader_election.cc:288] T 00000000000000000000000000000000 P 4d154bf2e9be473ab119c009a8ac51e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260928 14:47:15.997139 31536 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d154bf2e9be473ab119c009a8ac51e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:47:15.997284 9349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:47:15.997404 31531 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d154bf2e9be473ab119c009a8ac51e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:47:15.997478 31536 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d154bf2e9be473ab119c009a8ac51e2 [term 1 LEADER]: Becoming Leader. State: Replica: 4d154bf2e9be473ab119c009a8ac51e2, State: Running, Role: LEADER
I20260928 14:47:15.997700 31536 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d154bf2e9be473ab119c009a8ac51e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d154bf2e9be473ab119c009a8ac51e2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 34295 } }
I20260928 14:47:15.998189 31537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d154bf2e9be473ab119c009a8ac51e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d154bf2e9be473ab119c009a8ac51e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d154bf2e9be473ab119c009a8ac51e2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 34295 } } }
I20260928 14:47:15.998270 31537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d154bf2e9be473ab119c009a8ac51e2 [sys.catalog]: This master's current role is: LEADER
I20260928 14:47:15.998174 31536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d154bf2e9be473ab119c009a8ac51e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d154bf2e9be473ab119c009a8ac51e2. Latest consensus state: current_term: 1 leader_uuid: "4d154bf2e9be473ab119c009a8ac51e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d154bf2e9be473ab119c009a8ac51e2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 34295 } } }
I20260928 14:47:15.998472 31536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d154bf2e9be473ab119c009a8ac51e2 [sys.catalog]: This master's current role is: LEADER
W20260928 14:47:15.999214 31555 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4d154bf2e9be473ab119c009a8ac51e2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:47:15.999275 31555 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:47:15.999336 31546 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:47:15.999533 31546 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:47:16.000422 31535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:15.999056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33777 (local address 127.9.33.126:34295)
0928 14:47:15.999092 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:15.999094 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:15.999379 (+ 285us) server_negotiation.cc:407] Connection header received
0928 14:47:15.999463 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:15.999464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:15.999470 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:15.999524 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:16.000203 (+ 679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:16.000204 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:16.000205 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0928 14:47:16.000209 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:16.000225 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0928 14:47:16.000230 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:16.000276 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:16.000338 (+ 62us) server_negotiation.cc:299] Negotiation successful
0928 14:47:16.000354 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260928 14:47:16.001338 31546 catalog_manager.cc:1347] Generated new cluster ID: c7122b0eec0e4f9eb8802c80017298b5
I20260928 14:47:16.001401 31546 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:47:16.014137 31546 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:47:16.014364 31546 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:47:16.017552 31546 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d154bf2e9be473ab119c009a8ac51e2: Generated new TSK 0
I20260928 14:47:16.017812 31546 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260928 14:47:16.025008 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 31463
2026-09-28T14:47:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1327 ms)
[----------] 18 tests from AuthTokenIssuingTest (24126 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:47:16 dist-test-slave-pv0w krb5kdc[31562](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:47:16 dist-test-slave-pv0w krb5kdc[31562](info): set up 2 sockets
Sep 28 14:47:16 dist-test-slave-pv0w krb5kdc[31562](info): commencing operation
krb5kdc: starting...
W20260928 14:47:17.068928 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:47:17 dist-test-slave-pv0w krb5kdc[31562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:47:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:47:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.33.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
I20260928 14:47:17.195807 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:40605
--webserver_interface=127.9.33.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:45371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:40605,127.9.33.125:36147,127.9.33.124:38269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:47:17.282660 31575 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 14:47:17.282899 31575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:47:17.282917 31575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:47:17.282939 31575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:47:17.284170 31575 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:47:17.284217 31575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:47:17.284232 31575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:47:17.284245 31575 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:47:17.284258 31575 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:47:17.285743 31575 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:45371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:40605,127.9.33.125:36147,127.9.33.124:38269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:40605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:47:17.285948 31575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:47:17.286129 31575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:47:17.291368 31581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:47:17.291375 31580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:47:17.291407 31575 server_base.cc:1034] running on GCE node
W20260928 14:47:17.291384 31583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:47:17.291791 31575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:47:17.292133 31575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:47:17.293309 31575 hybrid_clock.cc:648] HybridClock initialized: now 1790606837293288 us; error 39 us; skew 500 ppm
Sep 28 14:47:17 dist-test-slave-pv0w krb5kdc[31562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606837, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:47:17.296012 31575 init.cc:377] Logged in from keytab as kudu/127.9.33.126@KRBTEST.COM (short username kudu)
I20260928 14:47:17.296420 31575 webserver.cc:460] Webserver started at http://127.9.33.126:33923/ using document root <none> and password file <none>
I20260928 14:47:17.296659 31575 fs_manager.cc:359] Metadata directory not provided
I20260928 14:47:17.296707 31575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:47:17.296861 31575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:47:17.298009 31575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "5db778b4c7a441a59154de20916288f2"
format_stamp: "Formatted at 2026-09-28 14:47:17 on dist-test-slave-pv0w"
server_key: "ef5e8501def06e071be87904c67f6bfd"
server_key_iv: "2cc2665872f7ba09ab4df47c771c076d"
server_key_version: "encryptionkey@0"
I20260928 14:47:17.298400 31575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "5db778b4c7a441a59154de20916288f2"
format_stamp: "Formatted at 2026-09-28 14:47:17 on dist-test-slave-pv0w"
server_key: "ef5e8501def06e071be87904c67f6bfd"
server_key_iv: "2cc2665872f7ba09ab4df47c771c076d"
server_key_version: "encryptionkey@0"
I20260928 14:47:17.299856 31575 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:47:17.300686 31590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:47:17.300940 31575 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:47:17.301051 31575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "5db778b4c7a441a59154de20916288f2"
format_stamp: "Formatted at 2026-09-28 14:47:17 on dist-test-slave-pv0w"
server_key: "ef5e8501def06e071be87904c67f6bfd"
server_key_iv: "2cc2665872f7ba09ab4df47c771c076d"
server_key_version: "encryptionkey@0"
I20260928 14:47:17.301154 31575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:47:17.308049 31575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:47:17.309980 31575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:47:17.310149 31575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:47:17.314076 31575 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:40605
I20260928 14:47:17.314102 31642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:40605 every 8 connection(s)
I20260928 14:47:17.314460 31575 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:47:17.315516 31643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:47:17.317204 31643 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40605 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:47:17.321731 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 31575
I20260928 14:47:17.321868 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:47:17.322083 9349 external_mini_cluster.cc:1428] Setting key c574af2bf4da442d31c2532eec5541d7
Sep 28 14:47:17 dist-test-slave-pv0w krb5kdc[31562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606837, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:47:17.328488 31645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.317847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:40605 (local address 127.0.0.1:46272)
0928 14:47:17.318017 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:17.318056 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0928 14:47:17.318522 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:47:17.318842 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:47:17.318844 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:47:17.319103 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:47:17.319276 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:47:17.319280 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.320397 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.320398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:47:17.321124 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:47:17.321126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.321693 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.321695 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:47:17.321787 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:17.322392 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:47:17.322404 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:47:17.324383 (+ 1979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:47:17.326824 (+ 2441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:47:17.326831 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:47:17.326832 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:47:17.327040 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:47:17.327333 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:47:17.327334 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:47:17.327334 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:47:17.327423 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:47:17.327946 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:47:17.327953 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:47:17.328017 (+ 64us) client_negotiation.cc:769] Sending connection context
0928 14:47:17.328106 (+ 89us) client_negotiation.cc:240] Negotiation successful
0928 14:47:17.328271 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":70,"threads_started":1}
I20260928 14:47:17.328579 31646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.318005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46272 (local address 127.9.33.126:40605)
0928 14:47:17.318134 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:17.318138 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:17.318168 (+ 30us) server_negotiation.cc:407] Connection header received
0928 14:47:17.318718 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:17.318720 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:17.318737 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:17.318814 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:17.319369 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.320317 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.321205 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.321535 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.321615 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:17.324502 (+ 2887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:17.324510 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:17.324511 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:47:17.324517 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:17.326739 (+ 2222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:17.327135 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:17.327139 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:17.327140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:17.327257 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:17.327497 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:17.327498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:17.327499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:17.327860 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:17.327959 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:17.328174 (+ 215us) server_negotiation.cc:299] Negotiation successful
0928 14:47:17.328368 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":70,"server-negotiator.queue_time_us":91,"thread_start_us":64,"threads_started":1}
I20260928 14:47:17.329631 31643 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36147 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:47:17.330024 31645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.329901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.125:36147 (local address 127.0.0.1:56106)
0928 14:47:17.329952 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:17.329999 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.125:36147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260928 14:47:17.330184 31592 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.33.125:36147: connect: Connection refused (error 111)
I20260928 14:47:17.330628 31645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.330532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.125:36147 (local address 127.0.0.1:56112)
0928 14:47:17.330578 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:17.330610 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.125:36147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260928 14:47:17.330745 31643 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.33.125:36147: Network error: Client connection negotiation failed: client connection to 127.9.33.125:36147: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.9.33.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.125.keytab.
Entry for principal kudu/127.9.33.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.125.keytab.
I20260928 14:47:17.351874 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.125:36147
--webserver_interface=127.9.33.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.125.keytab
--principal=kudu/127.9.33.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:45371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:40605,127.9.33.125:36147,127.9.33.124:38269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:47:17.379997 31643 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36147 } attempt: 1
I20260928 14:47:17.380540 31645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.380388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.125:36147 (local address 127.0.0.1:56120)
0928 14:47:17.380459 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:17.380503 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.125:36147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260928 14:47:17.380889 31645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.380772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.125:36147 (local address 127.0.0.1:56132)
0928 14:47:17.380816 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:17.380872 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.125:36147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260928 14:47:17.380971 31643 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.33.125:36147: Network error: Client connection negotiation failed: client connection to 127.9.33.125:36147: connect: Connection refused (error 111)
W20260928 14:47:17.435786 31650 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 14:47:17.436051 31650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:47:17.436084 31650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:47:17.436125 31650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:47:17.437290 31650 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:47:17.437369 31650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:47:17.437392 31650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:47:17.437412 31650 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:47:17.437443 31650 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:47:17.439055 31650 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:45371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:40605,127.9.33.125:36147,127.9.33.124:38269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.125.keytab
--principal=kudu/127.9.33.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.125:36147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.33.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:47:17.439350 31650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:47:17.439608 31650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:47:17.444746 31658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:47:17.444753 31656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:47:17.444896 31650 server_base.cc:1034] running on GCE node
W20260928 14:47:17.444983 31655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:47:17.445254 31650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:47:17.445533 31650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:47:17.446682 31650 hybrid_clock.cc:648] HybridClock initialized: now 1790606837446670 us; error 30 us; skew 500 ppm
Sep 28 14:47:17 dist-test-slave-pv0w krb5kdc[31562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606837, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:47:17.449105 31643 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36147 } attempt: 2
I20260928 14:47:17.449355 31650 init.cc:377] Logged in from keytab as kudu/127.9.33.125@KRBTEST.COM (short username kudu)
I20260928 14:47:17.449536 31645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.449406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.125:36147 (local address 127.0.0.1:56140)
0928 14:47:17.449468 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:17.449505 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.125:36147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260928 14:47:17.449883 31645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.449786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.125:36147 (local address 127.0.0.1:56156)
0928 14:47:17.449835 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:17.449860 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.125:36147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260928 14:47:17.449928 31650 webserver.cc:460] Webserver started at http://127.9.33.125:44633/ using document root <none> and password file <none>
W20260928 14:47:17.449992 31643 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.33.125:36147: Network error: Client connection negotiation failed: client connection to 127.9.33.125:36147: connect: Connection refused (error 111)
I20260928 14:47:17.450220 31650 fs_manager.cc:359] Metadata directory not provided
I20260928 14:47:17.450271 31650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:47:17.450460 31650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:47:17.451584 31650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-1/data/instance:
uuid: "524126c517974c1686bd39e264d4c34a"
format_stamp: "Formatted at 2026-09-28 14:47:17 on dist-test-slave-pv0w"
server_key: "31bd2ed3c3e40ad030c9565a685fd88f"
server_key_iv: "d8a6b9508619740ca14e4270d527b255"
server_key_version: "encryptionkey@0"
I20260928 14:47:17.452000 31650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-1/wal/instance:
uuid: "524126c517974c1686bd39e264d4c34a"
format_stamp: "Formatted at 2026-09-28 14:47:17 on dist-test-slave-pv0w"
server_key: "31bd2ed3c3e40ad030c9565a685fd88f"
server_key_iv: "d8a6b9508619740ca14e4270d527b255"
server_key_version: "encryptionkey@0"
I20260928 14:47:17.453473 31650 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:47:17.454317 31665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:47:17.454581 31650 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:47:17.454687 31650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-1/wal
uuid: "524126c517974c1686bd39e264d4c34a"
format_stamp: "Formatted at 2026-09-28 14:47:17 on dist-test-slave-pv0w"
server_key: "31bd2ed3c3e40ad030c9565a685fd88f"
server_key_iv: "d8a6b9508619740ca14e4270d527b255"
server_key_version: "encryptionkey@0"
I20260928 14:47:17.454771 31650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:47:17.481900 31650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:47:17.484126 31650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:47:17.484316 31650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:47:17.488690 31650 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.125:36147
I20260928 14:47:17.488775 31717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.125:36147 every 8 connection(s)
I20260928 14:47:17.489149 31650 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-1/data/info.pb
I20260928 14:47:17.490160 31718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:47:17.491818 31718 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40605 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:47:17.497303 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 31650
I20260928 14:47:17.497418 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-1/wal/instance
Sep 28 14:47:17 dist-test-slave-pv0w krb5kdc[31562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606837, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.125@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:47:17.497678 9349 external_mini_cluster.cc:1428] Setting key 1b9704f9e9ce20fa1ae37c704275f2a5
I20260928 14:47:17.501012 31646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.492571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46284 (local address 127.9.33.126:40605)
0928 14:47:17.492654 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:17.492656 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:17.492845 (+ 189us) server_negotiation.cc:407] Connection header received
0928 14:47:17.493305 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:17.493307 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:17.493317 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:17.493365 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:17.493959 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.494444 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.495452 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.495661 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.495760 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:17.498116 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:17.498118 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:17.498118 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:47:17.498126 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:17.499564 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:17.500027 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:17.500029 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:17.500029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:17.500092 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:17.500358 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:17.500359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:17.500359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:17.500480 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:17.500557 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:17.500833 (+ 276us) server_negotiation.cc:299] Negotiation successful
0928 14:47:17.500919 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260928 14:47:17.501243 31720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.492549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:40605 (local address 127.0.0.1:46284)
0928 14:47:17.492722 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:17.492759 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 14:47:17.493222 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:47:17.493425 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:47:17.493428 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:47:17.493688 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:47:17.493889 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:47:17.493893 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.494523 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.494523 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:47:17.495360 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:47:17.495362 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.495810 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.495811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:47:17.495917 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:17.496474 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:47:17.496482 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:47:17.498016 (+ 1534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:47:17.499677 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:47:17.499686 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:47:17.499688 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:47:17.499948 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:47:17.500166 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:47:17.500166 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:47:17.500167 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:47:17.500288 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:47:17.500549 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:47:17.500556 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:47:17.500702 (+ 146us) client_negotiation.cc:769] Sending connection context
0928 14:47:17.500812 (+ 110us) client_negotiation.cc:240] Negotiation successful
0928 14:47:17.500973 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":84,"threads_started":1}
I20260928 14:47:17.501945 31718 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36147 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.33.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.124@KRBTEST.COM" created.
Sep 28 14:47:17 dist-test-slave-pv0w krb5kdc[31562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606837, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.125@KRBTEST.COM for kudu/127.9.33.125@KRBTEST.COM
I20260928 14:47:17.511587 31720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.502228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.125:36147 (local address 127.0.0.1:56168)
0928 14:47:17.502284 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:17.502296 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:47:17.502341 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:47:17.502603 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:47:17.502604 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:47:17.502772 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:47:17.502817 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:47:17.502818 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.503684 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.503685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:47:17.504315 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:47:17.504316 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.504835 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.504836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:47:17.504905 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:17.505341 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:47:17.505346 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:47:17.508286 (+ 2940us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:47:17.510481 (+ 2195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:47:17.510482 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:47:17.510482 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:47:17.510654 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:47:17.510868 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:47:17.510869 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:47:17.510869 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:47:17.510925 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:47:17.511240 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:47:17.511241 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:47:17.511280 (+ 39us) client_negotiation.cc:769] Sending connection context
0928 14:47:17.511345 (+ 65us) client_negotiation.cc:240] Negotiation successful
0928 14:47:17.511451 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260928 14:47:17.511610 31722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.502311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56168 (local address 127.9.33.125:36147)
0928 14:47:17.502475 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:17.502479 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:17.502494 (+ 15us) server_negotiation.cc:407] Connection header received
0928 14:47:17.502510 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:17.502512 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:17.502523 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:17.502603 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:17.502900 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.503582 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.504406 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.504766 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.504838 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:17.508389 (+ 3551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:17.508399 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:17.508399 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:47:17.508406 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:17.510397 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:17.510738 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:17.510741 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:17.510742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:17.510801 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:17.511006 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:17.511007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:17.511007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:17.511170 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:17.511262 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:17.511374 (+ 112us) server_negotiation.cc:299] Negotiation successful
0928 14:47:17.511502 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":140,"thread_start_us":89,"threads_started":1}
I20260928 14:47:17.512285 31718 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.33.124" port: 38269 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:47:17.512672 31720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.512548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:38269 (local address 127.0.0.1:57452)
0928 14:47:17.512597 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:17.512640 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.124:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260928 14:47:17.512836 31667 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.33.124:38269: connect: Connection refused (error 111)
I20260928 14:47:17.513398 31720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.513275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:38269 (local address 127.0.0.1:57464)
0928 14:47:17.513343 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:17.513370 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.124:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260928 14:47:17.513525 31718 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.33.124:38269: Network error: Client connection negotiation failed: client connection to 127.9.33.124:38269: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.124.keytab.
Entry for principal kudu/127.9.33.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.124.keytab.
I20260928 14:47:17.526751 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.124:38269
--webserver_interface=127.9.33.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.124.keytab
--principal=kudu/127.9.33.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:45371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:40605,127.9.33.125:36147,127.9.33.124:38269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:47:17.541167 31643 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36147 } attempt: 3
Sep 28 14:47:17 dist-test-slave-pv0w krb5kdc[31562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606837, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for kudu/127.9.33.125@KRBTEST.COM
I20260928 14:47:17.548753 31645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.541608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.125:36147 (local address 127.0.0.1:56184)
0928 14:47:17.541683 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:17.541698 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 14:47:17.541776 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:47:17.541949 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:47:17.541950 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:47:17.542118 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:47:17.542185 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:47:17.542187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.542905 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.542906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:47:17.543442 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:47:17.543444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.543927 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.543928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:47:17.544045 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:17.544551 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:47:17.544556 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:47:17.545879 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:47:17.547472 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:47:17.547473 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:47:17.547473 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:47:17.547703 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:47:17.548025 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:47:17.548026 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:47:17.548027 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:47:17.548099 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:47:17.548422 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:47:17.548423 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:47:17.548475 (+ 52us) client_negotiation.cc:769] Sending connection context
0928 14:47:17.548545 (+ 70us) client_negotiation.cc:240] Negotiation successful
0928 14:47:17.548651 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260928 14:47:17.548785 31722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.541694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56184 (local address 127.9.33.125:36147)
0928 14:47:17.541762 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:17.541765 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:17.541777 (+ 12us) server_negotiation.cc:407] Connection header received
0928 14:47:17.541846 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:17.541848 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:17.541867 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:17.541933 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:17.542271 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.542807 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.543535 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.543834 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.543925 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:17.545986 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:17.545988 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:17.545988 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:47:17.545995 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:17.547387 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:17.547836 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:17.547838 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:17.547838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:17.547910 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:17.548211 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:17.548212 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:17.548213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:17.548348 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:17.548413 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:17.548629 (+ 216us) server_negotiation.cc:299] Negotiation successful
0928 14:47:17.548718 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260928 14:47:17.549263 31643 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.33.124" port: 38269 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:47:17.549638 31645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.549527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:38269 (local address 127.0.0.1:57466)
0928 14:47:17.549584 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:17.549615 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.124:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260928 14:47:17.550035 31645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.549927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:38269 (local address 127.0.0.1:57472)
0928 14:47:17.549954 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:17.549978 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.124:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260928 14:47:17.550143 31643 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.33.124:38269: Network error: Client connection negotiation failed: client connection to 127.9.33.124:38269: connect: Connection refused (error 111)
I20260928 14:47:17.562760 31718 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.33.124" port: 38269 } attempt: 1
I20260928 14:47:17.563282 31720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.563137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:38269 (local address 127.0.0.1:57480)
0928 14:47:17.563207 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:17.563249 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.124:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260928 14:47:17.563635 31720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.563532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:38269 (local address 127.0.0.1:57484)
0928 14:47:17.563580 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:17.563607 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.124:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260928 14:47:17.563719 31718 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.33.124:38269: Network error: Client connection negotiation failed: client connection to 127.9.33.124:38269: connect: Connection refused (error 111)
I20260928 14:47:17.581305 31643 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.33.124" port: 38269 } attempt: 1
I20260928 14:47:17.581881 31645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.581728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:38269 (local address 127.0.0.1:57496)
0928 14:47:17.581804 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:17.581849 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.124:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260928 14:47:17.582273 31645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.582181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:38269 (local address 127.0.0.1:57498)
0928 14:47:17.582229 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:17.582255 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.124:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260928 14:47:17.582383 31643 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.33.124:38269: Network error: Client connection negotiation failed: client connection to 127.9.33.124:38269: connect: Connection refused (error 111)
W20260928 14:47:17.613376 31725 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 14:47:17.613673 31725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:47:17.613706 31725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:47:17.613754 31725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:47:17.615008 31725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:47:17.615092 31725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:47:17.615114 31725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:47:17.615142 31725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:47:17.615163 31725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:47:17.616829 31725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:45371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:40605,127.9.33.125:36147,127.9.33.124:38269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.124.keytab
--principal=kudu/127.9.33.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.124:38269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.33.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:47:17.617151 31725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:47:17.617409 31725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:47:17.622733 31731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:47:17.622733 31730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:47:17.622759 31733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:47:17.623114 31725 server_base.cc:1034] running on GCE node
I20260928 14:47:17.623361 31725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:47:17.623637 31725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:47:17.624810 31725 hybrid_clock.cc:648] HybridClock initialized: now 1790606837624792 us; error 35 us; skew 500 ppm
Sep 28 14:47:17 dist-test-slave-pv0w krb5kdc[31562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606837, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:47:17.627573 31725 init.cc:377] Logged in from keytab as kudu/127.9.33.124@KRBTEST.COM (short username kudu)
I20260928 14:47:17.628161 31725 webserver.cc:460] Webserver started at http://127.9.33.124:43341/ using document root <none> and password file <none>
I20260928 14:47:17.628404 31725 fs_manager.cc:359] Metadata directory not provided
I20260928 14:47:17.628451 31725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:47:17.628589 31725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:47:17.629850 31725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-2/data/instance:
uuid: "e328fb6c40344f89897eb095c3c377a3"
format_stamp: "Formatted at 2026-09-28 14:47:17 on dist-test-slave-pv0w"
server_key: "ad50ccae8e563f3b8986a57bcc7eef13"
server_key_iv: "6bdf6baedb9c878dbd1948c442309ed0"
server_key_version: "encryptionkey@0"
I20260928 14:47:17.630262 31725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-2/wal/instance:
uuid: "e328fb6c40344f89897eb095c3c377a3"
format_stamp: "Formatted at 2026-09-28 14:47:17 on dist-test-slave-pv0w"
server_key: "ad50ccae8e563f3b8986a57bcc7eef13"
server_key_iv: "6bdf6baedb9c878dbd1948c442309ed0"
server_key_version: "encryptionkey@0"
I20260928 14:47:17.631767 31725 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260928 14:47:17.631927 31718 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.33.124" port: 38269 } attempt: 2
I20260928 14:47:17.632400 31720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.632237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:38269 (local address 127.0.0.1:57502)
0928 14:47:17.632301 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:17.632358 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.124:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260928 14:47:17.632781 31740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:47:17.632810 31720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.632705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:38269 (local address 127.0.0.1:57504)
0928 14:47:17.632757 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:17.632788 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.124:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260928 14:47:17.632916 31718 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.33.124:38269: Network error: Client connection negotiation failed: client connection to 127.9.33.124:38269: connect: Connection refused (error 111)
I20260928 14:47:17.632952 31725 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:47:17.633050 31725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-2/wal
uuid: "e328fb6c40344f89897eb095c3c377a3"
format_stamp: "Formatted at 2026-09-28 14:47:17 on dist-test-slave-pv0w"
server_key: "ad50ccae8e563f3b8986a57bcc7eef13"
server_key_iv: "6bdf6baedb9c878dbd1948c442309ed0"
server_key_version: "encryptionkey@0"
I20260928 14:47:17.633111 31725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:47:17.655318 31725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:47:17.657573 31643 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.33.124" port: 38269 } attempt: 2
I20260928 14:47:17.657960 31725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:47:17.658094 31645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.657907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:38269 (local address 127.0.0.1:57518)
0928 14:47:17.657998 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:17.658039 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.124:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260928 14:47:17.658167 31725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:47:17.658494 31645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.658384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:38269 (local address 127.0.0.1:57534)
0928 14:47:17.658436 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:17.658469 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.124:38269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260928 14:47:17.658588 31643 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.33.124:38269: Network error: Client connection negotiation failed: client connection to 127.9.33.124:38269: connect: Connection refused (error 111)
I20260928 14:47:17.662436 31725 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.124:38269
I20260928 14:47:17.662467 31792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.124:38269 every 8 connection(s)
I20260928 14:47:17.662860 31725 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-2/data/info.pb
I20260928 14:47:17.663872 31793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:47:17.665472 31793 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40605 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 14:47:17 dist-test-slave-pv0w krb5kdc[31562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606837, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.124@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:47:17.672463 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 31725
I20260928 14:47:17.672580 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790606561545179-9349-0/minicluster-data/master-2/wal/instance
I20260928 14:47:17.672811 9349 external_mini_cluster.cc:1428] Setting key 877ae684a47c1511a3ac8f51e654c539
I20260928 14:47:17.675326 31646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.666254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46288 (local address 127.9.33.126:40605)
0928 14:47:17.666319 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:17.666321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:17.666524 (+ 203us) server_negotiation.cc:407] Connection header received
0928 14:47:17.666987 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:17.666988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:17.666998 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:17.667062 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:17.667812 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.668297 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.669323 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.669507 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.669565 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:17.671963 (+ 2398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:17.671964 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:17.671965 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:47:17.671972 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:17.673935 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:17.674345 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:17.674346 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:17.674346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:17.674414 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:17.674682 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:17.674683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:17.674683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:17.674802 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:17.674857 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:17.675189 (+ 332us) server_negotiation.cc:299] Negotiation successful
0928 14:47:17.675259 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260928 14:47:17.675586 31795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.666175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:40605 (local address 127.0.0.1:46288)
0928 14:47:17.666395 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:17.666437 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0928 14:47:17.666900 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:47:17.667169 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:47:17.667173 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:47:17.667430 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:47:17.667664 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:47:17.667668 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.668384 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.668385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:47:17.669240 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:47:17.669243 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.669647 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.669648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:47:17.669761 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:17.670286 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:47:17.670302 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:47:17.671860 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:47:17.674021 (+ 2161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:47:17.674024 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:47:17.674026 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:47:17.674264 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:47:17.674480 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:47:17.674481 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:47:17.674481 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:47:17.674572 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:47:17.674870 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:47:17.674876 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:47:17.675037 (+ 161us) client_negotiation.cc:769] Sending connection context
0928 14:47:17.675163 (+ 126us) client_negotiation.cc:240] Negotiation successful
0928 14:47:17.675336 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":104,"threads_started":1}
I20260928 14:47:17.676411 31793 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36147 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 14:47:17 dist-test-slave-pv0w krb5kdc[31562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.124@KRBTEST.COM
Sep 28 14:47:17 dist-test-slave-pv0w krb5kdc[31562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606837, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.124@KRBTEST.COM for kudu/127.9.33.125@KRBTEST.COM
I20260928 14:47:17.682647 31796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.673420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57548 (local address 127.9.33.124:38269)
0928 14:47:17.673639 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:17.673644 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:17.673714 (+ 70us) server_negotiation.cc:407] Connection header received
0928 14:47:17.673791 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:17.673793 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:17.673806 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:17.673875 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:17.674624 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.675690 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.676686 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.677087 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.677189 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:17.679507 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:17.679518 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:17.679519 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:47:17.679528 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:17.681362 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:17.681747 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:17.681756 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:17.681756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:17.681815 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:17.682030 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:17.682031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:17.682031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:17.682190 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:17.682351 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:17.682426 (+ 75us) server_negotiation.cc:299] Negotiation successful
0928 14:47:17.682512 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":118,"threads_started":1}
I20260928 14:47:17.683946 31722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.676846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56196 (local address 127.9.33.125:36147)
0928 14:47:17.676915 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:17.676918 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:17.676927 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:47:17.676952 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:17.676953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:17.676962 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:17.677038 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:17.677332 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.677973 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.678698 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.678923 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.679011 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:17.681243 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:17.681245 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:17.681245 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:47:17.681263 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:17.682747 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:17.683095 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:17.683096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:17.683097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:17.683158 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:17.683371 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:17.683372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:17.683378 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:17.683524 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:17.683587 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:17.683809 (+ 222us) server_negotiation.cc:299] Negotiation successful
0928 14:47:17.683883 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260928 14:47:17.683980 31795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.676771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.125:36147 (local address 127.0.0.1:56196)
0928 14:47:17.676832 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:17.676842 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:47:17.676883 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:47:17.677043 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:47:17.677044 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:47:17.677201 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:47:17.677251 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:47:17.677253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.678085 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.678087 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:47:17.678610 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:47:17.678611 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.679012 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.679013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:47:17.679092 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:17.679527 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:47:17.679532 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:47:17.681130 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:47:17.682832 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:47:17.682833 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:47:17.682834 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:47:17.683012 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:47:17.683227 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:47:17.683227 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:47:17.683228 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:47:17.683301 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:47:17.683618 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:47:17.683619 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:47:17.683667 (+ 48us) client_negotiation.cc:769] Sending connection context
0928 14:47:17.683756 (+ 89us) client_negotiation.cc:240] Negotiation successful
0928 14:47:17.683855 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260928 14:47:17.684443 31793 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.33.124" port: 38269 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 14:47:17 dist-test-slave-pv0w krb5kdc[31562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606837, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.124@KRBTEST.COM for kudu/127.9.33.124@KRBTEST.COM
Sep 28 14:47:17 dist-test-slave-pv0w krb5kdc[31562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:47:17.691928 31795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.684747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:38269 (local address 127.0.0.1:57552)
0928 14:47:17.684779 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:17.684789 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:47:17.684826 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:47:17.684948 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:47:17.684949 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:47:17.685089 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:47:17.685117 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:47:17.685119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.686007 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.686008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:47:17.686707 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:47:17.686709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.687071 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.687071 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:47:17.687122 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:17.687521 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:47:17.687525 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:47:17.688851 (+ 1326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:47:17.690769 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:47:17.690771 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:47:17.690771 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:47:17.690993 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:47:17.691206 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:47:17.691207 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:47:17.691207 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:47:17.691276 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:47:17.691549 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:47:17.691550 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:47:17.691606 (+ 56us) client_negotiation.cc:769] Sending connection context
0928 14:47:17.691661 (+ 55us) client_negotiation.cc:240] Negotiation successful
0928 14:47:17.691795 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
I20260928 14:47:17.692019 31796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.684786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57552 (local address 127.9.33.124:38269)
0928 14:47:17.684820 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:17.684822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:17.684830 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:47:17.684883 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:17.684883 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:17.684893 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:17.684945 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:17.685199 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.685922 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.686784 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.686992 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.687071 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:17.688954 (+ 1883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:17.688955 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:17.688956 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:47:17.688962 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:17.690682 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:17.691075 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:17.691076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:17.691077 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:17.691124 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:17.691342 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:17.691343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:17.691343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:17.691447 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:17.691522 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:17.691709 (+ 187us) server_negotiation.cc:299] Negotiation successful
0928 14:47:17.691849 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 14:47:17.692934 31646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.683528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46302 (local address 127.9.33.126:40605)
0928 14:47:17.683568 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:17.683570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:17.683579 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:47:17.683659 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:17.683660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:17.683669 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:17.683703 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:17.684656 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.685197 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.686171 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.686473 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.686548 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:17.689534 (+ 2986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:17.689535 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:17.689536 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:47:17.689544 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:17.691447 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:17.691924 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:17.691925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:17.691925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:17.691995 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:17.692233 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:17.692234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:17.692234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:17.692372 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:17.692554 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:17.692688 (+ 134us) server_negotiation.cc:299] Negotiation successful
0928 14:47:17.692800 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 14:47:17.693732 31793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e328fb6c40344f89897eb095c3c377a3: Bootstrap starting.
I20260928 14:47:17.694588 31793 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e328fb6c40344f89897eb095c3c377a3: Neither blocks nor log segments found. Creating new log.
I20260928 14:47:17.694886 31793 log.cc:824] T 00000000000000000000000000000000 P e328fb6c40344f89897eb095c3c377a3: Log is configured to *not* fsync() on all Append() calls
I20260928 14:47:17.695849 31793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e328fb6c40344f89897eb095c3c377a3: No bootstrap required, opened a new log
I20260928 14:47:17.697422 31793 raft_consensus.cc:348] T 00000000000000000000000000000000 P e328fb6c40344f89897eb095c3c377a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db778b4c7a441a59154de20916288f2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40605 } } peers { permanent_uuid: "524126c517974c1686bd39e264d4c34a" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36147 } } peers { permanent_uuid: "e328fb6c40344f89897eb095c3c377a3" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 38269 } }
I20260928 14:47:17.697568 31793 raft_consensus.cc:729] T 00000000000000000000000000000000 P e328fb6c40344f89897eb095c3c377a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e328fb6c40344f89897eb095c3c377a3, State: Initialized, Role: FOLLOWER
I20260928 14:47:17.697672 31793 consensus_queue.cc:260] T 00000000000000000000000000000000 P e328fb6c40344f89897eb095c3c377a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db778b4c7a441a59154de20916288f2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40605 } } peers { permanent_uuid: "524126c517974c1686bd39e264d4c34a" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36147 } } peers { permanent_uuid: "e328fb6c40344f89897eb095c3c377a3" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 38269 } }
I20260928 14:47:17.698021 31793 sys_catalog.cc:564] T 00000000000000000000000000000000 P e328fb6c40344f89897eb095c3c377a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:47:17.697969 31799 sys_catalog.cc:455] T 00000000000000000000000000000000 P e328fb6c40344f89897eb095c3c377a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db778b4c7a441a59154de20916288f2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40605 } } peers { permanent_uuid: "524126c517974c1686bd39e264d4c34a" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36147 } } peers { permanent_uuid: "e328fb6c40344f89897eb095c3c377a3" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 38269 } } }
I20260928 14:47:17.698108 31799 sys_catalog.cc:458] T 00000000000000000000000000000000 P e328fb6c40344f89897eb095c3c377a3 [sys.catalog]: This master's current role is: FOLLOWER
W20260928 14:47:17.700264 31810 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e328fb6c40344f89897eb095c3c377a3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:47:17.700379 31810 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Sep 28 14:47:17 dist-test-slave-pv0w krb5kdc[31562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606837, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.125@KRBTEST.COM
I20260928 14:47:17.700423 31725 master_runner.cc:186] Error getting master registration for 127.9.33.126:40605: OK, instance_id { permanent_uuid: "5db778b4c7a441a59154de20916288f2" instance_seqno: 1790606837312958 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 14:47:17.703266 31725 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 14:47:17.704690 31722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.695982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56210 (local address 127.9.33.125:36147)
0928 14:47:17.696027 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:17.696029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:17.696133 (+ 104us) server_negotiation.cc:407] Connection header received
0928 14:47:17.696209 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:17.696211 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:17.696220 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:17.696275 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:17.696772 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.697255 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.698166 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.698429 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.698495 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:17.701543 (+ 3048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:17.701546 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:17.701546 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:47:17.701555 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:17.703372 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:17.703875 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:17.703876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:17.703876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:17.703935 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:17.704163 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:17.704163 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:17.704163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:17.704284 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:17.704429 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:17.704527 (+ 98us) server_negotiation.cc:299] Negotiation successful
0928 14:47:17.704605 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 14:47:17.706393 9349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:47:17.707644 31617 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "5db778b4c7a441a59154de20916288f2"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46302
I20260928 14:47:17.724057 31718 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.33.124" port: 38269 } attempt: 3
Sep 28 14:47:17 dist-test-slave-pv0w krb5kdc[31562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606837, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.125@KRBTEST.COM for kudu/127.9.33.124@KRBTEST.COM
I20260928 14:47:17.732764 31720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.724475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:38269 (local address 127.0.0.1:57560)
0928 14:47:17.724556 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:17.724571 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0928 14:47:17.724633 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:47:17.724791 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:47:17.724792 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:47:17.725001 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:47:17.725061 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:47:17.725063 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.726003 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.726005 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:47:17.727326 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:47:17.727329 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.727829 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.727830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:47:17.727898 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:17.728481 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:47:17.728488 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:47:17.729860 (+ 1372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:47:17.731568 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:47:17.731569 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:47:17.731570 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:47:17.731807 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:47:17.732047 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:47:17.732048 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:47:17.732048 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:47:17.732113 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:47:17.732403 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:47:17.732404 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:47:17.732451 (+ 47us) client_negotiation.cc:769] Sending connection context
0928 14:47:17.732528 (+ 77us) client_negotiation.cc:240] Negotiation successful
0928 14:47:17.732647 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260928 14:47:17.733053 31796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.724561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57560 (local address 127.9.33.124:38269)
0928 14:47:17.724657 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:17.724659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:17.724671 (+ 12us) server_negotiation.cc:407] Connection header received
0928 14:47:17.724706 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:17.724707 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:17.724715 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:17.724765 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:17.725160 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.725902 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.727425 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.727755 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.727822 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:17.729997 (+ 2175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:17.729999 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:17.729999 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:47:17.730007 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:17.731460 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:17.731899 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:17.731901 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:17.731901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:17.731968 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:17.732176 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:17.732177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:17.732178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:17.732330 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:17.732411 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:17.732901 (+ 490us) server_negotiation.cc:299] Negotiation successful
0928 14:47:17.732994 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260928 14:47:17.734368 31718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 524126c517974c1686bd39e264d4c34a: Bootstrap starting.
I20260928 14:47:17.735086 31718 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 524126c517974c1686bd39e264d4c34a: Neither blocks nor log segments found. Creating new log.
I20260928 14:47:17.735380 31718 log.cc:824] T 00000000000000000000000000000000 P 524126c517974c1686bd39e264d4c34a: Log is configured to *not* fsync() on all Append() calls
I20260928 14:47:17.736294 31718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 524126c517974c1686bd39e264d4c34a: No bootstrap required, opened a new log
I20260928 14:47:17.737926 31718 raft_consensus.cc:348] T 00000000000000000000000000000000 P 524126c517974c1686bd39e264d4c34a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db778b4c7a441a59154de20916288f2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40605 } } peers { permanent_uuid: "524126c517974c1686bd39e264d4c34a" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36147 } } peers { permanent_uuid: "e328fb6c40344f89897eb095c3c377a3" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 38269 } }
I20260928 14:47:17.738067 31718 raft_consensus.cc:729] T 00000000000000000000000000000000 P 524126c517974c1686bd39e264d4c34a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 524126c517974c1686bd39e264d4c34a, State: Initialized, Role: FOLLOWER
I20260928 14:47:17.738153 31718 consensus_queue.cc:260] T 00000000000000000000000000000000 P 524126c517974c1686bd39e264d4c34a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db778b4c7a441a59154de20916288f2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40605 } } peers { permanent_uuid: "524126c517974c1686bd39e264d4c34a" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36147 } } peers { permanent_uuid: "e328fb6c40344f89897eb095c3c377a3" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 38269 } }
I20260928 14:47:17.738451 31813 sys_catalog.cc:455] T 00000000000000000000000000000000 P 524126c517974c1686bd39e264d4c34a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db778b4c7a441a59154de20916288f2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40605 } } peers { permanent_uuid: "524126c517974c1686bd39e264d4c34a" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36147 } } peers { permanent_uuid: "e328fb6c40344f89897eb095c3c377a3" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 38269 } } }
I20260928 14:47:17.738528 31718 sys_catalog.cc:564] T 00000000000000000000000000000000 P 524126c517974c1686bd39e264d4c34a [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:47:17.738543 31813 sys_catalog.cc:458] T 00000000000000000000000000000000 P 524126c517974c1686bd39e264d4c34a [sys.catalog]: This master's current role is: FOLLOWER
W20260928 14:47:17.740290 31824 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 524126c517974c1686bd39e264d4c34a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:47:17.740346 31824 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:47:17.740576 31650 master_runner.cc:186] Error getting master registration for 127.9.33.126:40605: OK, instance_id { permanent_uuid: "5db778b4c7a441a59154de20916288f2" instance_seqno: 1790606837312958 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 14:47:17.743566 31650 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 14:47:17.757737 31643 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.33.124" port: 38269 } attempt: 3
Sep 28 14:47:17 dist-test-slave-pv0w krb5kdc[31562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606837, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for kudu/127.9.33.124@KRBTEST.COM
I20260928 14:47:17.766356 31796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.758216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57572 (local address 127.9.33.124:38269)
0928 14:47:17.758264 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:17.758267 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:17.758279 (+ 12us) server_negotiation.cc:407] Connection header received
0928 14:47:17.758313 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:17.758314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:17.758324 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:17.758399 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:17.758741 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.759520 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.760627 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.760948 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.761029 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:17.763406 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:17.763408 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:17.763408 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:47:17.763417 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:17.765111 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:17.765529 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:17.765531 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:17.765531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:17.765595 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:17.765803 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:17.765804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:17.765804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:17.765959 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:17.766023 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:17.766178 (+ 155us) server_negotiation.cc:299] Negotiation successful
0928 14:47:17.766261 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260928 14:47:17.766355 31645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.758121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:38269 (local address 127.0.0.1:57572)
0928 14:47:17.758183 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:17.758195 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0928 14:47:17.758253 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:47:17.758408 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:47:17.758409 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:47:17.758607 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:47:17.758665 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:47:17.758668 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.759629 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.759631 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:47:17.760528 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:47:17.760532 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.761046 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.761047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:47:17.761112 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:17.761645 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:47:17.761650 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:47:17.763298 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:47:17.765207 (+ 1909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:47:17.765209 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:47:17.765209 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:47:17.765448 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:47:17.765673 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:47:17.765673 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:47:17.765674 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:47:17.765737 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:47:17.766020 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:47:17.766021 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:47:17.766071 (+ 50us) client_negotiation.cc:769] Sending connection context
0928 14:47:17.766139 (+ 68us) client_negotiation.cc:240] Negotiation successful
0928 14:47:17.766240 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260928 14:47:17.768812 31643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5db778b4c7a441a59154de20916288f2: Bootstrap starting.
I20260928 14:47:17.769600 31643 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5db778b4c7a441a59154de20916288f2: Neither blocks nor log segments found. Creating new log.
I20260928 14:47:17.769919 31643 log.cc:824] T 00000000000000000000000000000000 P 5db778b4c7a441a59154de20916288f2: Log is configured to *not* fsync() on all Append() calls
I20260928 14:47:17.770798 31643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5db778b4c7a441a59154de20916288f2: No bootstrap required, opened a new log
I20260928 14:47:17.772267 31643 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5db778b4c7a441a59154de20916288f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db778b4c7a441a59154de20916288f2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40605 } } peers { permanent_uuid: "524126c517974c1686bd39e264d4c34a" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36147 } } peers { permanent_uuid: "e328fb6c40344f89897eb095c3c377a3" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 38269 } }
I20260928 14:47:17.772419 31643 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5db778b4c7a441a59154de20916288f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5db778b4c7a441a59154de20916288f2, State: Initialized, Role: FOLLOWER
I20260928 14:47:17.772572 31643 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5db778b4c7a441a59154de20916288f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db778b4c7a441a59154de20916288f2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40605 } } peers { permanent_uuid: "524126c517974c1686bd39e264d4c34a" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36147 } } peers { permanent_uuid: "e328fb6c40344f89897eb095c3c377a3" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 38269 } }
I20260928 14:47:17.772974 31617 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5db778b4c7a441a59154de20916288f2 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260928 14:47:17.772933 31827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5db778b4c7a441a59154de20916288f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db778b4c7a441a59154de20916288f2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40605 } } peers { permanent_uuid: "524126c517974c1686bd39e264d4c34a" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36147 } } peers { permanent_uuid: "e328fb6c40344f89897eb095c3c377a3" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 38269 } } }
I20260928 14:47:17.773108 31617 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5db778b4c7a441a59154de20916288f2 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:47:17.773134 31827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5db778b4c7a441a59154de20916288f2 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:47:17.773890 31617 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5db778b4c7a441a59154de20916288f2 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db778b4c7a441a59154de20916288f2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40605 } } peers { permanent_uuid: "524126c517974c1686bd39e264d4c34a" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36147 } } peers { permanent_uuid: "e328fb6c40344f89897eb095c3c377a3" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 38269 } }
I20260928 14:47:17.774130 31617 leader_election.cc:288] T 00000000000000000000000000000000 P 5db778b4c7a441a59154de20916288f2 [CANDIDATE]: Term 1 election: Requested vote from peers 524126c517974c1686bd39e264d4c34a (127.9.33.125:36147), e328fb6c40344f89897eb095c3c377a3 (127.9.33.124:38269)
I20260928 14:47:17.774513 31692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5db778b4c7a441a59154de20916288f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "524126c517974c1686bd39e264d4c34a"
I20260928 14:47:17.774645 31692 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 524126c517974c1686bd39e264d4c34a [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:47:17.772943 31643 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5db778b4c7a441a59154de20916288f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:47:17.775491 31692 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 524126c517974c1686bd39e264d4c34a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5db778b4c7a441a59154de20916288f2 in term 1.
I20260928 14:47:17.775779 31592 leader_election.cc:302] T 00000000000000000000000000000000 P 5db778b4c7a441a59154de20916288f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 524126c517974c1686bd39e264d4c34a, 5db778b4c7a441a59154de20916288f2; no voters:
I20260928 14:47:17.776111 31767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5db778b4c7a441a59154de20916288f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e328fb6c40344f89897eb095c3c377a3"
I20260928 14:47:17.776243 31767 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e328fb6c40344f89897eb095c3c377a3 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:47:17.777045 31767 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e328fb6c40344f89897eb095c3c377a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5db778b4c7a441a59154de20916288f2 in term 1.
I20260928 14:47:17.777359 31827 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5db778b4c7a441a59154de20916288f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:47:17.777539 31827 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5db778b4c7a441a59154de20916288f2 [term 1 LEADER]: Becoming Leader. State: Replica: 5db778b4c7a441a59154de20916288f2, State: Running, Role: LEADER
I20260928 14:47:17.777650 31827 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5db778b4c7a441a59154de20916288f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db778b4c7a441a59154de20916288f2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40605 } } peers { permanent_uuid: "524126c517974c1686bd39e264d4c34a" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36147 } } peers { permanent_uuid: "e328fb6c40344f89897eb095c3c377a3" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 38269 } }
W20260928 14:47:17.777370 31842 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5db778b4c7a441a59154de20916288f2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:47:17.777933 31842 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:47:17.779237 31575 master_runner.cc:263] No leader master found from master 5db778b4c7a441a59154de20916288f2
I20260928 14:47:17.779284 31575 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 14:47:17.779624 31827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5db778b4c7a441a59154de20916288f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5db778b4c7a441a59154de20916288f2. Latest consensus state: current_term: 1 leader_uuid: "5db778b4c7a441a59154de20916288f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db778b4c7a441a59154de20916288f2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40605 } } peers { permanent_uuid: "524126c517974c1686bd39e264d4c34a" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36147 } } peers { permanent_uuid: "e328fb6c40344f89897eb095c3c377a3" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 38269 } } }
I20260928 14:47:17.779774 31827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5db778b4c7a441a59154de20916288f2 [sys.catalog]: This master's current role is: LEADER
I20260928 14:47:17.780062 31846 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:47:17.780303 31846 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:47:17.782347 31767 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e328fb6c40344f89897eb095c3c377a3 [term 1 FOLLOWER]: Refusing update from remote peer 5db778b4c7a441a59154de20916288f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:47:17.782660 31827 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5db778b4c7a441a59154de20916288f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e328fb6c40344f89897eb095c3c377a3" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 38269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:47:17.783869 31799 sys_catalog.cc:455] T 00000000000000000000000000000000 P e328fb6c40344f89897eb095c3c377a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5db778b4c7a441a59154de20916288f2. Latest consensus state: current_term: 1 leader_uuid: "5db778b4c7a441a59154de20916288f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db778b4c7a441a59154de20916288f2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40605 } } peers { permanent_uuid: "524126c517974c1686bd39e264d4c34a" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36147 } } peers { permanent_uuid: "e328fb6c40344f89897eb095c3c377a3" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 38269 } } }
I20260928 14:47:17.783982 31799 sys_catalog.cc:458] T 00000000000000000000000000000000 P e328fb6c40344f89897eb095c3c377a3 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:47:17.784379 31692 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 524126c517974c1686bd39e264d4c34a [term 1 FOLLOWER]: Refusing update from remote peer 5db778b4c7a441a59154de20916288f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:47:17.784650 31827 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5db778b4c7a441a59154de20916288f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "524126c517974c1686bd39e264d4c34a" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:47:17.785899 31813 sys_catalog.cc:455] T 00000000000000000000000000000000 P 524126c517974c1686bd39e264d4c34a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5db778b4c7a441a59154de20916288f2. Latest consensus state: current_term: 1 leader_uuid: "5db778b4c7a441a59154de20916288f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db778b4c7a441a59154de20916288f2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40605 } } peers { permanent_uuid: "524126c517974c1686bd39e264d4c34a" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36147 } } peers { permanent_uuid: "e328fb6c40344f89897eb095c3c377a3" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 38269 } } }
I20260928 14:47:17.786002 31813 sys_catalog.cc:458] T 00000000000000000000000000000000 P 524126c517974c1686bd39e264d4c34a [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:47:17.787496 31827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5db778b4c7a441a59154de20916288f2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5db778b4c7a441a59154de20916288f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db778b4c7a441a59154de20916288f2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40605 } } peers { permanent_uuid: "524126c517974c1686bd39e264d4c34a" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36147 } } peers { permanent_uuid: "e328fb6c40344f89897eb095c3c377a3" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 38269 } } }
I20260928 14:47:17.787578 31827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5db778b4c7a441a59154de20916288f2 [sys.catalog]: This master's current role is: LEADER
I20260928 14:47:17.788074 31813 sys_catalog.cc:455] T 00000000000000000000000000000000 P 524126c517974c1686bd39e264d4c34a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5db778b4c7a441a59154de20916288f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db778b4c7a441a59154de20916288f2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40605 } } peers { permanent_uuid: "524126c517974c1686bd39e264d4c34a" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36147 } } peers { permanent_uuid: "e328fb6c40344f89897eb095c3c377a3" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 38269 } } }
I20260928 14:47:17.788160 31813 sys_catalog.cc:458] T 00000000000000000000000000000000 P 524126c517974c1686bd39e264d4c34a [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:47:17.788407 31850 mvcc.cc:205] Tried to move back new op lower bound from 7334325607554129920 to 7334325607544180736. Current Snapshot: MvccSnapshot[applied={T|T < 7334325607554129920}]
I20260928 14:47:17.788621 31853 mvcc.cc:205] Tried to move back new op lower bound from 7334325607554129920 to 7334325607544180736. Current Snapshot: MvccSnapshot[applied={T|T < 7334325607554129920}]
I20260928 14:47:17.789494 31827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5db778b4c7a441a59154de20916288f2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5db778b4c7a441a59154de20916288f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db778b4c7a441a59154de20916288f2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40605 } } peers { permanent_uuid: "524126c517974c1686bd39e264d4c34a" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36147 } } peers { permanent_uuid: "e328fb6c40344f89897eb095c3c377a3" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 38269 } } }
I20260928 14:47:17.789584 31827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5db778b4c7a441a59154de20916288f2 [sys.catalog]: This master's current role is: LEADER
I20260928 14:47:17.789981 31799 sys_catalog.cc:455] T 00000000000000000000000000000000 P e328fb6c40344f89897eb095c3c377a3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5db778b4c7a441a59154de20916288f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5db778b4c7a441a59154de20916288f2" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 40605 } } peers { permanent_uuid: "524126c517974c1686bd39e264d4c34a" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 36147 } } peers { permanent_uuid: "e328fb6c40344f89897eb095c3c377a3" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 38269 } } }
I20260928 14:47:17.790077 31799 sys_catalog.cc:458] T 00000000000000000000000000000000 P e328fb6c40344f89897eb095c3c377a3 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:47:17.791147 31851 mvcc.cc:205] Tried to move back new op lower bound from 7334325607554129920 to 7334325607544180736. Current Snapshot: MvccSnapshot[applied={T|T < 7334325607554129920}]
I20260928 14:47:17.796190 31846 catalog_manager.cc:1347] Generated new cluster ID: 89e6499911c648f5b3cd90d313eb87b3
I20260928 14:47:17.796253 31846 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:47:17.805714 31722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.779984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56218 (local address 127.9.33.125:36147)
0928 14:47:17.780031 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:17.780034 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:17.788909 (+ 8875us) server_negotiation.cc:407] Connection header received
0928 14:47:17.789117 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:17.789118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:17.789129 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:17.789157 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:17.792608 (+ 3451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.793306 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.796177 (+ 2871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.796487 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.796532 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:17.799392 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:17.799394 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:17.799394 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:47:17.799402 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:17.801101 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:17.803745 (+ 2644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:17.803746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:17.803747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:17.803810 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:17.804784 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:17.804786 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:17.804786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:17.804932 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:17.805016 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:17.805523 (+ 507us) server_negotiation.cc:299] Negotiation successful
0928 14:47:17.805615 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260928 14:47:17.807221 31796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.780566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57588 (local address 127.9.33.124:38269)
0928 14:47:17.780611 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:17.780613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:17.789012 (+ 8399us) server_negotiation.cc:407] Connection header received
0928 14:47:17.789970 (+ 958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:17.789971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:17.789981 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:17.790011 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:17.791848 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.792511 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.795238 (+ 2727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.795551 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.795594 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:17.801624 (+ 6030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:17.801626 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:17.801627 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:47:17.801635 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:17.803287 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:17.805209 (+ 1922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:17.805211 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:17.805211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:17.805274 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:17.806582 (+ 1308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:17.806583 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:17.806584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:17.806729 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:17.806822 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:17.807051 (+ 229us) server_negotiation.cc:299] Negotiation successful
0928 14:47:17.807133 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260928 14:47:17.813050 31646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:17.780247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46318 (local address 127.9.33.126:40605)
0928 14:47:17.780290 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:17.780292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:17.784003 (+ 3711us) server_negotiation.cc:407] Connection header received
0928 14:47:17.787859 (+ 3856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:17.787860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:17.787871 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:17.787902 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:17.790760 (+ 2858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.791475 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.808064 (+ 16589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:17.808357 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:17.808409 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:17.810028 (+ 1619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:17.810030 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:17.810030 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:47:17.810038 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:17.811736 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:17.812152 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:17.812154 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:17.812154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:17.812213 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:17.812427 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:17.812428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:17.812428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:17.812574 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:17.812647 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:17.812877 (+ 230us) server_negotiation.cc:299] Negotiation successful
0928 14:47:17.812959 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 14:47:17.839339 31846 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:47:17.840169 31846 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:47:17.853179 31846 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5db778b4c7a441a59154de20916288f2: Generated new TSK 0
I20260928 14:47:17.853420 31846 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 14:47:17.884770 31646 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:47:17.883098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46332 (local address 127.9.33.126:40605)
0928 14:47:17.883146 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:17.883149 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:17.884085 (+ 936us) server_negotiation.cc:407] Connection header received
0928 14:47:17.884187 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:17.884188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:17.884200 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:17.884229 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:17.884692 (+ 463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46332: BlockingRecv error: recv got EOF from 127.0.0.1:46332 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260928 14:47:17.885504 31722 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:47:17.883487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56228 (local address 127.9.33.125:36147)
0928 14:47:17.883530 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:17.883532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:17.885014 (+ 1482us) server_negotiation.cc:407] Connection header received
0928 14:47:17.885107 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:17.885108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:17.885118 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:17.885146 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:17.885427 (+ 281us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56228: BlockingRecv error: recv got EOF from 127.0.0.1:56228 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260928 14:47:17.886190 31796 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:47:17.883827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57598 (local address 127.9.33.124:38269)
0928 14:47:17.883865 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:17.883867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:17.885692 (+ 1825us) server_negotiation.cc:407] Connection header received
0928 14:47:17.885809 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:17.885810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:17.885821 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:17.885849 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:17.886127 (+ 278us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57598: BlockingRecv error: recv got EOF from 127.0.0.1:57598 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260928 14:47:17.888736 31722 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:47:17.887942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56232 (local address 127.9.33.125:36147)
0928 14:47:17.887978 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:17.887980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:17.888174 (+ 194us) server_negotiation.cc:407] Connection header received
0928 14:47:17.888272 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:17.888273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:17.888289 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:17.888317 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:17.888688 (+ 371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56232: BlockingRecv error: recv got EOF from 127.0.0.1:56232 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260928 14:47:17.891885 31722 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:47:17.891093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56246 (local address 127.9.33.125:36147)
0928 14:47:17.891125 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:17.891127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:17.891350 (+ 223us) server_negotiation.cc:407] Connection header received
0928 14:47:17.891440 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:17.891441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:17.891450 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:17.891498 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:17.891843 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56246: BlockingRecv error: recv got EOF from 127.0.0.1:56246 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260928 14:47:17.895684 31722 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:47:17.894974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56254 (local address 127.9.33.125:36147)
0928 14:47:17.895008 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:17.895010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:17.895202 (+ 192us) server_negotiation.cc:407] Connection header received
0928 14:47:17.895288 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:17.895289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:17.895298 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:17.895324 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:17.895640 (+ 316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56254: BlockingRecv error: recv got EOF from 127.0.0.1:56254 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260928 14:47:17.901473 31722 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:47:17.900870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56256 (local address 127.9.33.125:36147)
0928 14:47:17.900908 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:17.900910 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:17.901012 (+ 102us) server_negotiation.cc:407] Connection header received
0928 14:47:17.901072 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:17.901072 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:17.901083 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:17.901131 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:17.901415 (+ 284us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56256: BlockingRecv error: recv got EOF from 127.0.0.1:56256 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260928 14:47:17.911201 31722 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:47:17.910469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56268 (local address 127.9.33.125:36147)
0928 14:47:17.910512 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:17.910514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:17.910665 (+ 151us) server_negotiation.cc:407] Connection header received
0928 14:47:17.910737 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:17.910738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:17.910749 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:17.910823 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:17.911132 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56268: BlockingRecv error: recv got EOF from 127.0.0.1:56268 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260928 14:47:17.929126 31722 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:47:17.928362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56270 (local address 127.9.33.125:36147)
0928 14:47:17.928409 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:17.928411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:17.928551 (+ 140us) server_negotiation.cc:407] Connection header received
0928 14:47:17.928605 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:17.928607 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:17.928620 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:17.928693 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:17.929044 (+ 351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56270: BlockingRecv error: recv got EOF from 127.0.0.1:56270 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260928 14:47:17.963052 31722 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:47:17.962354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56282 (local address 127.9.33.125:36147)
0928 14:47:17.962413 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:17.962415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:17.962514 (+ 99us) server_negotiation.cc:407] Connection header received
0928 14:47:17.962549 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:17.962550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:17.962563 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:17.962643 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:17.962979 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56282: BlockingRecv error: recv got EOF from 127.0.0.1:56282 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260928 14:47:18.029217 31722 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:47:18.028362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56284 (local address 127.9.33.125:36147)
0928 14:47:18.028432 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:18.028436 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:18.028471 (+ 35us) server_negotiation.cc:407] Connection header received
0928 14:47:18.028667 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:18.028669 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:18.028687 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:18.028765 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:18.029087 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56284: BlockingRecv error: recv got EOF from 127.0.0.1:56284 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260928 14:47:18.159410 31722 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:47:18.158627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56286 (local address 127.9.33.125:36147)
0928 14:47:18.158707 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:18.158709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:18.158827 (+ 118us) server_negotiation.cc:407] Connection header received
0928 14:47:18.158908 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:18.158909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:18.158923 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:18.159012 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:18.159362 (+ 350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56286: BlockingRecv error: recv got EOF from 127.0.0.1:56286 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260928 14:47:18.417742 31722 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:47:18.417005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56292 (local address 127.9.33.125:36147)
0928 14:47:18.417091 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:18.417093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:18.417102 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:47:18.417129 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:18.417130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:18.417141 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:18.417206 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:18.417643 (+ 437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56292: BlockingRecv error: recv got EOF from 127.0.0.1:56292 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260928 14:47:18.700881 31810 catalog_manager.cc:1259] Loaded cluster ID: 89e6499911c648f5b3cd90d313eb87b3
I20260928 14:47:18.700973 31810 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e328fb6c40344f89897eb095c3c377a3: loading cluster ID for follower catalog manager: success
I20260928 14:47:18.701756 31810 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e328fb6c40344f89897eb095c3c377a3: acquiring CA information for follower catalog manager: success
I20260928 14:47:18.702020 31810 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e328fb6c40344f89897eb095c3c377a3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 14:47:18.740813 31824 catalog_manager.cc:1259] Loaded cluster ID: 89e6499911c648f5b3cd90d313eb87b3
I20260928 14:47:18.740898 31824 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 524126c517974c1686bd39e264d4c34a: loading cluster ID for follower catalog manager: success
I20260928 14:47:18.741811 31824 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 524126c517974c1686bd39e264d4c34a: acquiring CA information for follower catalog manager: success
I20260928 14:47:18.742153 31824 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 524126c517974c1686bd39e264d4c34a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 14:47:18.933694 31924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:18.930966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56306 (local address 127.9.33.125:36147)
0928 14:47:18.931075 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:18.931077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:18.931086 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:47:18.931136 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:18.931137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:18.931190 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:18.931272 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:47:18.931485 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:18.932034 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:18.932807 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:18.933024 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:18.933091 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:18.933342 (+ 251us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:47:18.933460 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:47:18.933523 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:18.933573 (+ 50us) server_negotiation.cc:299] Negotiation successful
0928 14:47:18.933592 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":60,"threads_started":1}
I20260928 14:47:18.938601 31930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:18.935965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57606 (local address 127.9.33.124:38269)
0928 14:47:18.936155 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:18.936157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:18.936165 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:47:18.936202 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:18.936202 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:18.936241 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:18.936314 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:47:18.936403 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:18.936974 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:18.937825 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:18.938110 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:18.938165 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:18.938302 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:47:18.938416 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:47:18.938474 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:18.938508 (+ 34us) server_negotiation.cc:299] Negotiation successful
0928 14:47:18.938533 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":114,"threads_started":1}
I20260928 14:47:18.943565 31924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:18.940816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56310 (local address 127.9.33.125:36147)
0928 14:47:18.940858 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:18.940859 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:18.940867 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:47:18.940894 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:18.940895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:18.940903 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:18.940957 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:47:18.941107 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:18.941895 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:18.942743 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:18.942999 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:18.943065 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:18.943247 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:47:18.943281 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:47:18.943340 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:18.943435 (+ 95us) server_negotiation.cc:299] Negotiation successful
0928 14:47:18.943460 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 14:47:18.943701 31930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:18.940777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57616 (local address 127.9.33.124:38269)
0928 14:47:18.940814 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:18.940816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:18.940913 (+ 97us) server_negotiation.cc:407] Connection header received
0928 14:47:18.940995 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:18.940996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:18.941005 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:18.941035 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:47:18.941189 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:18.941683 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:18.942799 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:18.943105 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:18.943170 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:18.943391 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:47:18.943424 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:47:18.943473 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:18.943598 (+ 125us) server_negotiation.cc:299] Negotiation successful
0928 14:47:18.943626 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260928 14:47:18.944149 31937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:18.940545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46348 (local address 127.9.33.126:40605)
0928 14:47:18.940670 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:18.940672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:18.940682 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:47:18.940777 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:18.940780 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:18.940998 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:18.941105 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:47:18.941230 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:18.941978 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:18.942981 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:18.943286 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:18.943347 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:18.943603 (+ 256us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:47:18.943912 (+ 309us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:47:18.943981 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:18.944036 (+ 55us) server_negotiation.cc:299] Negotiation successful
0928 14:47:18.944061 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":60,"threads_started":1}
I20260928 14:47:18.945880 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 31575
I20260928 14:47:18.951033 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 31650
I20260928 14:47:18.956233 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 31725
2026-09-28T14:47:18Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2932 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 28 14:47:19 dist-test-slave-pv0w krb5kdc[31941](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 28 14:47:19 dist-test-slave-pv0w krb5kdc[31941](info): set up 2 sockets
Sep 28 14:47:19 dist-test-slave-pv0w krb5kdc[31941](info): commencing operation
krb5kdc: starting...
W20260928 14:47:19.996798 9349 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/krb5kdc/test-user.keytab.
Sep 28 14:47:20 dist-test-slave-pv0w krb5kdc[31941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-28T14:47:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-28T14:47:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.33.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
Entry for principal kudu/127.9.33.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab.
I20260928 14:47:20.136106 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:35843
--webserver_interface=127.9.33.126
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:38581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:35843,127.9.33.125:45123,127.9.33.124:46023
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260928 14:47:20.219076 31954 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 14:47:20.219369 31954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:47:20.219401 31954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:47:20.219450 31954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:47:20.220709 31954 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:47:20.220788 31954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:47:20.220811 31954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:47:20.220834 31954 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:47:20.220861 31954 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:47:20.222534 31954 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:38581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:35843,127.9.33.125:45123,127.9.33.124:46023
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.126.keytab
--principal=kudu/127.9.33.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.126:35843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.33.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:47:20.222849 31954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:47:20.223130 31954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:47:20.228544 31960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:47:20.228549 31962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:47:20.228544 31959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:47:20.228579 31954 server_base.cc:1034] running on GCE node
I20260928 14:47:20.229033 31954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:47:20.229305 31954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:47:20.230463 31954 hybrid_clock.cc:648] HybridClock initialized: now 1790606840230439 us; error 40 us; skew 500 ppm
Sep 28 14:47:20 dist-test-slave-pv0w krb5kdc[31941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606840, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:47:20.233155 31954 init.cc:377] Logged in from keytab as kudu/127.9.33.126@KRBTEST.COM (short username kudu)
I20260928 14:47:20.233604 31954 webserver.cc:460] Webserver started at http://127.9.33.126:43329/ using document root <none> and password file <none>
I20260928 14:47:20.233798 31954 fs_manager.cc:359] Metadata directory not provided
I20260928 14:47:20.233839 31954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:47:20.233932 31954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:47:20.234954 31954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-0/data/instance:
uuid: "b7e580e5dc9c41a8a8107fd92e860e61"
format_stamp: "Formatted at 2026-09-28 14:47:20 on dist-test-slave-pv0w"
server_key: "9412770f66931c77d99cfc3e353bf87e"
server_key_iv: "42cbdb09f756b8f6a6ea50d514d9581e"
server_key_version: "encryptionkey@0"
I20260928 14:47:20.235275 31954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-0/wal/instance:
uuid: "b7e580e5dc9c41a8a8107fd92e860e61"
format_stamp: "Formatted at 2026-09-28 14:47:20 on dist-test-slave-pv0w"
server_key: "9412770f66931c77d99cfc3e353bf87e"
server_key_iv: "42cbdb09f756b8f6a6ea50d514d9581e"
server_key_version: "encryptionkey@0"
I20260928 14:47:20.236768 31954 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260928 14:47:20.237528 31969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:47:20.237741 31954 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260928 14:47:20.237854 31954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-0/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-0/wal
uuid: "b7e580e5dc9c41a8a8107fd92e860e61"
format_stamp: "Formatted at 2026-09-28 14:47:20 on dist-test-slave-pv0w"
server_key: "9412770f66931c77d99cfc3e353bf87e"
server_key_iv: "42cbdb09f756b8f6a6ea50d514d9581e"
server_key_version: "encryptionkey@0"
I20260928 14:47:20.237946 31954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:47:20.255718 31954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:47:20.258193 31954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:47:20.258370 31954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:47:20.262578 31954 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.126:35843
I20260928 14:47:20.262616 32021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.126:35843 every 8 connection(s)
I20260928 14:47:20.263013 31954 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-0/data/info.pb
I20260928 14:47:20.264137 32022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:47:20.265766 32022 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35843 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:47:20.271332 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 31954
I20260928 14:47:20.271411 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-0/wal/instance
I20260928 14:47:20.271626 9349 external_mini_cluster.cc:1428] Setting key be385d254cb9365df3b6d6141f11d254
Sep 28 14:47:20 dist-test-slave-pv0w krb5kdc[31941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606840, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:47:20.275753 32025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.266497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50042 (local address 127.9.33.126:35843)
0928 14:47:20.266648 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:20.266657 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:20.266745 (+ 88us) server_negotiation.cc:407] Connection header received
0928 14:47:20.267231 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:20.267233 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:20.267257 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:20.267344 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:20.267951 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.268772 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.269625 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.269933 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.270017 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:20.272270 (+ 2253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:20.272279 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:20.272279 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:47:20.272287 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:20.273981 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:20.274411 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:20.274414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:20.274415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:20.274524 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:20.274766 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:20.274767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:20.274768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:20.275074 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:20.275165 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:20.275349 (+ 184us) server_negotiation.cc:299] Negotiation successful
0928 14:47:20.275511 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":111,"thread_start_us":87,"threads_started":1}
I20260928 14:47:20.275763 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.266438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:35843 (local address 127.0.0.1:50042)
0928 14:47:20.266606 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:20.266647 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0928 14:47:20.267108 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:47:20.267372 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:47:20.267374 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:47:20.267626 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:47:20.267865 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:47:20.267869 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.268861 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.268862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:47:20.269545 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:47:20.269547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.270030 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.270031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:47:20.270103 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:20.270648 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:47:20.270656 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:47:20.272183 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:47:20.274070 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:47:20.274077 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:47:20.274079 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:47:20.274325 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:47:20.274618 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:47:20.274619 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:47:20.274619 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:47:20.274700 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:47:20.275146 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:47:20.275151 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:47:20.275206 (+ 55us) client_negotiation.cc:769] Sending connection context
0928 14:47:20.275280 (+ 74us) client_negotiation.cc:240] Negotiation successful
0928 14:47:20.275404 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":92,"threads_started":1}
I20260928 14:47:20.276556 32022 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.33.125" port: 45123 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:47:20.276921 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.276810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.125:45123 (local address 127.0.0.1:40126)
0928 14:47:20.276858 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:20.276901 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.125:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260928 14:47:20.277092 31970 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.33.125:45123: connect: Connection refused (error 111)
I20260928 14:47:20.277513 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.277430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.125:45123 (local address 127.0.0.1:40138)
0928 14:47:20.277474 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:20.277497 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.125:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260928 14:47:20.277647 32022 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.33.125:45123: Network error: Client connection negotiation failed: client connection to 127.9.33.125:45123: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.9.33.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.125.keytab.
Entry for principal kudu/127.9.33.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.125.keytab.
I20260928 14:47:20.299638 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.125:45123
--webserver_interface=127.9.33.125
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.125.keytab
--principal=kudu/127.9.33.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:38581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:35843,127.9.33.125:45123,127.9.33.124:46023
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:47:20.326912 32022 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.33.125" port: 45123 } attempt: 1
I20260928 14:47:20.327447 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.327302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.125:45123 (local address 127.0.0.1:40144)
0928 14:47:20.327372 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:20.327416 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.125:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260928 14:47:20.327906 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.327809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.125:45123 (local address 127.0.0.1:40152)
0928 14:47:20.327852 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:20.327881 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.125:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260928 14:47:20.328001 32022 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.33.125:45123: Network error: Client connection negotiation failed: client connection to 127.9.33.125:45123: connect: Connection refused (error 111)
W20260928 14:47:20.383853 32029 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 14:47:20.384117 32029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:47:20.384150 32029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:47:20.384197 32029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:47:20.385373 32029 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:47:20.385447 32029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:47:20.385470 32029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:47:20.385504 32029 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:47:20.385552 32029 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:47:20.387107 32029 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:38581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:35843,127.9.33.125:45123,127.9.33.124:46023
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.125.keytab
--principal=kudu/127.9.33.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.125:45123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.33.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:47:20.387398 32029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:47:20.387639 32029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:47:20.392680 32034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:47:20.392776 32035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:47:20.392699 32037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:47:20.393062 32029 server_base.cc:1034] running on GCE node
I20260928 14:47:20.393283 32029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:47:20.393558 32029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:47:20.394722 32029 hybrid_clock.cc:648] HybridClock initialized: now 1790606840394712 us; error 35 us; skew 500 ppm
I20260928 14:47:20.396150 32022 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.33.125" port: 45123 } attempt: 2
I20260928 14:47:20.396580 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.396440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.125:45123 (local address 127.0.0.1:40166)
0928 14:47:20.396508 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:20.396545 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.125:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260928 14:47:20.396981 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.396889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.125:45123 (local address 127.0.0.1:40170)
0928 14:47:20.396936 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:20.396961 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.125:45123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260928 14:47:20.397119 32022 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.33.125:45123: Network error: Client connection negotiation failed: client connection to 127.9.33.125:45123: connect: Connection refused (error 111)
Sep 28 14:47:20 dist-test-slave-pv0w krb5kdc[31941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606840, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:47:20.397812 32029 init.cc:377] Logged in from keytab as kudu/127.9.33.125@KRBTEST.COM (short username kudu)
I20260928 14:47:20.398283 32029 webserver.cc:460] Webserver started at http://127.9.33.125:39827/ using document root <none> and password file <none>
I20260928 14:47:20.398514 32029 fs_manager.cc:359] Metadata directory not provided
I20260928 14:47:20.398558 32029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:47:20.398705 32029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:47:20.399895 32029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-1/data/instance:
uuid: "d0edf43b2d484bf08df745bb30eebd3e"
format_stamp: "Formatted at 2026-09-28 14:47:20 on dist-test-slave-pv0w"
server_key: "d4b7b6e3eacf7e51f4304178a8e36b6d"
server_key_iv: "a9cd7842330a3357b3556b42a9fbe722"
server_key_version: "encryptionkey@0"
I20260928 14:47:20.400266 32029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-1/wal/instance:
uuid: "d0edf43b2d484bf08df745bb30eebd3e"
format_stamp: "Formatted at 2026-09-28 14:47:20 on dist-test-slave-pv0w"
server_key: "d4b7b6e3eacf7e51f4304178a8e36b6d"
server_key_iv: "a9cd7842330a3357b3556b42a9fbe722"
server_key_version: "encryptionkey@0"
I20260928 14:47:20.401655 32029 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260928 14:47:20.402508 32044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:47:20.402709 32029 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260928 14:47:20.402822 32029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-1/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-1/wal
uuid: "d0edf43b2d484bf08df745bb30eebd3e"
format_stamp: "Formatted at 2026-09-28 14:47:20 on dist-test-slave-pv0w"
server_key: "d4b7b6e3eacf7e51f4304178a8e36b6d"
server_key_iv: "a9cd7842330a3357b3556b42a9fbe722"
server_key_version: "encryptionkey@0"
I20260928 14:47:20.402897 32029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:47:20.419045 32029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:47:20.421276 32029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:47:20.421453 32029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:47:20.425697 32029 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.125:45123
I20260928 14:47:20.425730 32096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.125:45123 every 8 connection(s)
I20260928 14:47:20.426115 32029 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-1/data/info.pb
I20260928 14:47:20.427139 32097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:47:20.428885 32097 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35843 } has no permanent_uuid. Determining permanent_uuid...
Sep 28 14:47:20 dist-test-slave-pv0w krb5kdc[31941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606840, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.125@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:47:20.435309 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 32029
I20260928 14:47:20.435390 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-1/wal/instance
I20260928 14:47:20.435889 9349 external_mini_cluster.cc:1428] Setting key fe9d9cc9c0e5547bde1a6b5282c94147
I20260928 14:47:20.438122 32025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.429658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50056 (local address 127.9.33.126:35843)
0928 14:47:20.429731 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:20.429732 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:20.429950 (+ 218us) server_negotiation.cc:407] Connection header received
0928 14:47:20.430481 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:20.430483 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:20.430498 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:20.430549 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:20.431201 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.431702 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.432766 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.432982 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.433077 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:20.435223 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:20.435225 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:20.435225 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:47:20.435231 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:20.436613 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:20.437040 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:20.437041 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:20.437042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:20.437121 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:20.437452 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:20.437453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:20.437453 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:20.437570 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:20.437633 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:20.437968 (+ 335us) server_negotiation.cc:299] Negotiation successful
0928 14:47:20.438039 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260928 14:47:20.438380 32099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.429629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:35843 (local address 127.0.0.1:50056)
0928 14:47:20.429826 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:20.429863 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0928 14:47:20.430364 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:47:20.430649 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:47:20.430652 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:47:20.430916 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:47:20.431111 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:47:20.431115 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.431872 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.431873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:47:20.432673 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:47:20.432675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.433106 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.433108 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:47:20.433225 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:20.433743 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:47:20.433751 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:47:20.435122 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:47:20.436700 (+ 1578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:47:20.436707 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:47:20.436710 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:47:20.436932 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:47:20.437214 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:47:20.437216 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:47:20.437216 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:47:20.437375 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:47:20.437645 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:47:20.437652 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:47:20.437826 (+ 174us) client_negotiation.cc:769] Sending connection context
0928 14:47:20.437944 (+ 118us) client_negotiation.cc:240] Negotiation successful
0928 14:47:20.438123 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":94,"threads_started":1}
I20260928 14:47:20.438982 32097 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.33.125" port: 45123 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.33.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.33.124@KRBTEST.COM" created.
Sep 28 14:47:20 dist-test-slave-pv0w krb5kdc[31941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606840, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.125@KRBTEST.COM for kudu/127.9.33.125@KRBTEST.COM
I20260928 14:47:20.450775 32101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.439315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40172 (local address 127.9.33.125:45123)
0928 14:47:20.439468 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:20.439473 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:20.439496 (+ 23us) server_negotiation.cc:407] Connection header received
0928 14:47:20.439525 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:20.439527 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:20.439541 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:20.439613 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:20.439960 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.441193 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.442050 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.442574 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.442654 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:20.447773 (+ 5119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:20.447786 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:20.447786 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:47:20.447795 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:20.449519 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:20.449948 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:20.449951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:20.449951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:20.450006 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:20.450210 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:20.450210 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:20.450210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:20.450372 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:20.450435 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:20.450613 (+ 178us) server_negotiation.cc:299] Negotiation successful
0928 14:47:20.450692 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":95,"threads_started":1}
I20260928 14:47:20.450940 32099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.439247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.125:45123 (local address 127.0.0.1:40172)
0928 14:47:20.439294 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:20.439304 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:47:20.439336 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:47:20.439631 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:47:20.439632 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:47:20.439815 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:47:20.439880 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:47:20.439882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.441319 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.441320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:47:20.441887 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:47:20.441889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.442675 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.442676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:47:20.442738 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:20.443204 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:47:20.443208 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:47:20.447605 (+ 4397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:47:20.449604 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:47:20.449605 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:47:20.449606 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:47:20.449860 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:47:20.450073 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:47:20.450074 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:47:20.450074 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:47:20.450148 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:47:20.450438 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:47:20.450440 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:47:20.450494 (+ 54us) client_negotiation.cc:769] Sending connection context
0928 14:47:20.450576 (+ 82us) client_negotiation.cc:240] Negotiation successful
0928 14:47:20.450765 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":57}
I20260928 14:47:20.452921 32097 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.33.124" port: 46023 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:47:20.453347 32099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.453185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:46023 (local address 127.0.0.1:40728)
0928 14:47:20.453243 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:20.453308 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.124:46023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260928 14:47:20.453505 32045 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.33.124:46023: connect: Connection refused (error 111)
I20260928 14:47:20.453969 32099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.453851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:46023 (local address 127.0.0.1:40744)
0928 14:47:20.453907 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:20.453939 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.124:46023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260928 14:47:20.454114 32097 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.33.124:46023: Network error: Client connection negotiation failed: client connection to 127.9.33.124:46023: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.33.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.124.keytab.
Entry for principal kudu/127.9.33.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.124.keytab.
I20260928 14:47:20.466171 9349 external_mini_cluster.cc:1326] Running /tmp/dist-test-task4TnxN9/build/release/bin/kudu
/tmp/dist-test-task4TnxN9/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.124:46023
--webserver_interface=127.9.33.124
--webserver_port=0
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.124.keytab
--principal=kudu/127.9.33.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.33.84:38581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.33.126:35843,127.9.33.125:45123,127.9.33.124:46023
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260928 14:47:20.488319 32022 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.33.125" port: 45123 } attempt: 3
Sep 28 14:47:20 dist-test-slave-pv0w krb5kdc[31941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606840, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for kudu/127.9.33.125@KRBTEST.COM
I20260928 14:47:20.495643 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.488729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.125:45123 (local address 127.0.0.1:40184)
0928 14:47:20.488788 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:20.488808 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 14:47:20.488880 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:47:20.489042 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:47:20.489043 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:47:20.489219 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:47:20.489290 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:47:20.489292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.490036 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.490037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:47:20.490604 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:47:20.490606 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.491005 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.491006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:47:20.491096 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:20.491570 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:47:20.491580 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:47:20.492802 (+ 1222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:47:20.494495 (+ 1693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:47:20.494496 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:47:20.494496 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:47:20.494693 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:47:20.494917 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:47:20.494918 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:47:20.494919 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:47:20.494984 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:47:20.495265 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:47:20.495266 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:47:20.495320 (+ 54us) client_negotiation.cc:769] Sending connection context
0928 14:47:20.495404 (+ 84us) client_negotiation.cc:240] Negotiation successful
0928 14:47:20.495532 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260928 14:47:20.495643 32101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.488805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40184 (local address 127.9.33.125:45123)
0928 14:47:20.488878 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:20.488880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:20.488891 (+ 11us) server_negotiation.cc:407] Connection header received
0928 14:47:20.488946 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:20.488947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:20.488960 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:20.489034 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:20.489377 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.489939 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.490696 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.490925 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.491006 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:20.492893 (+ 1887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:20.492894 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:20.492895 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:47:20.492902 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:20.494371 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:20.494780 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:20.494781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:20.494782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:20.494846 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:20.495052 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:20.495053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:20.495053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:20.495181 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:20.495262 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:20.495443 (+ 181us) server_negotiation.cc:299] Negotiation successful
0928 14:47:20.495535 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260928 14:47:20.496097 32022 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.33.124" port: 46023 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:47:20.496455 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.496346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:46023 (local address 127.0.0.1:40748)
0928 14:47:20.496396 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:20.496427 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.124:46023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260928 14:47:20.496774 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.496680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:46023 (local address 127.0.0.1:40752)
0928 14:47:20.496730 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:20.496753 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.124:46023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260928 14:47:20.496870 32022 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.33.124:46023: Network error: Client connection negotiation failed: client connection to 127.9.33.124:46023: connect: Connection refused (error 111)
I20260928 14:47:20.503350 32097 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.33.124" port: 46023 } attempt: 1
I20260928 14:47:20.503852 32099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.503657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:46023 (local address 127.0.0.1:40754)
0928 14:47:20.503754 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:20.503806 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.124:46023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260928 14:47:20.504252 32099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.504125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:46023 (local address 127.0.0.1:40770)
0928 14:47:20.504184 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:20.504221 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.124:46023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260928 14:47:20.504386 32097 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.33.124:46023: Network error: Client connection negotiation failed: client connection to 127.9.33.124:46023: connect: Connection refused (error 111)
I20260928 14:47:20.528007 32022 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.33.124" port: 46023 } attempt: 1
I20260928 14:47:20.528558 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.528413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:46023 (local address 127.0.0.1:40784)
0928 14:47:20.528485 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:20.528528 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.124:46023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260928 14:47:20.528964 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.528876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:46023 (local address 127.0.0.1:40798)
0928 14:47:20.528921 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:20.528947 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.124:46023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260928 14:47:20.529074 32022 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.33.124:46023: Network error: Client connection negotiation failed: client connection to 127.9.33.124:46023: connect: Connection refused (error 111)
W20260928 14:47:20.552277 32104 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260928 14:47:20.552536 32104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260928 14:47:20.552577 32104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260928 14:47:20.552618 32104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260928 14:47:20.553802 32104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260928 14:47:20.553867 32104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260928 14:47:20.553893 32104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260928 14:47:20.553925 32104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260928 14:47:20.553960 32104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260928 14:47:20.555519 32104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.33.84:38581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.33.126:35843,127.9.33.125:45123,127.9.33.124:46023
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/krb5kdc/kudu_127.9.33.124.keytab
--principal=kudu/127.9.33.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.33.124:46023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.33.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 28 Sep 2026 13:47:06 UTC on ccycloud.quasar-fguyrz.root.comops.site
build id 4677365
I20260928 14:47:20.555840 32104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260928 14:47:20.556089 32104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260928 14:47:20.561043 32109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:47:20.561180 32104 server_base.cc:1034] running on GCE node
W20260928 14:47:20.561043 32110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260928 14:47:20.561157 32112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260928 14:47:20.561688 32104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260928 14:47:20.561957 32104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260928 14:47:20.563117 32104 hybrid_clock.cc:648] HybridClock initialized: now 1790606840563103 us; error 29 us; skew 500 ppm
Sep 28 14:47:20 dist-test-slave-pv0w krb5kdc[31941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606840, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260928 14:47:20.565779 32104 init.cc:377] Logged in from keytab as kudu/127.9.33.124@KRBTEST.COM (short username kudu)
I20260928 14:47:20.566268 32104 webserver.cc:460] Webserver started at http://127.9.33.124:46193/ using document root <none> and password file <none>
I20260928 14:47:20.566509 32104 fs_manager.cc:359] Metadata directory not provided
I20260928 14:47:20.566578 32104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260928 14:47:20.566744 32104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260928 14:47:20.567934 32104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-2/data/instance:
uuid: "a426bb520f00401d94b1aca8b5c77f30"
format_stamp: "Formatted at 2026-09-28 14:47:20 on dist-test-slave-pv0w"
server_key: "bab8504aa6fc1d6332878bdfc4627f6b"
server_key_iv: "6cb77bdbdb52915705c6e01440a5b319"
server_key_version: "encryptionkey@0"
I20260928 14:47:20.568310 32104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-2/wal/instance:
uuid: "a426bb520f00401d94b1aca8b5c77f30"
format_stamp: "Formatted at 2026-09-28 14:47:20 on dist-test-slave-pv0w"
server_key: "bab8504aa6fc1d6332878bdfc4627f6b"
server_key_iv: "6cb77bdbdb52915705c6e01440a5b319"
server_key_version: "encryptionkey@0"
I20260928 14:47:20.569777 32104 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260928 14:47:20.570541 32119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260928 14:47:20.570751 32104 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260928 14:47:20.570865 32104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-2/data,/tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-2/wal
uuid: "a426bb520f00401d94b1aca8b5c77f30"
format_stamp: "Formatted at 2026-09-28 14:47:20 on dist-test-slave-pv0w"
server_key: "bab8504aa6fc1d6332878bdfc4627f6b"
server_key_iv: "6cb77bdbdb52915705c6e01440a5b319"
server_key_version: "encryptionkey@0"
I20260928 14:47:20.570943 32104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260928 14:47:20.572558 32097 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.33.124" port: 46023 } attempt: 2
I20260928 14:47:20.572997 32099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.572872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:46023 (local address 127.0.0.1:40812)
0928 14:47:20.572935 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:20.572970 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.124:46023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260928 14:47:20.573371 32099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.573274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:46023 (local address 127.0.0.1:40826)
0928 14:47:20.573322 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:20.573352 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.124:46023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260928 14:47:20.573458 32097 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.33.124:46023: Network error: Client connection negotiation failed: client connection to 127.9.33.124:46023: connect: Connection refused (error 111)
I20260928 14:47:20.599514 32104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260928 14:47:20.601656 32104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260928 14:47:20.601835 32104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260928 14:47:20.604233 32022 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.33.124" port: 46023 } attempt: 2
I20260928 14:47:20.604772 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.604611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:46023 (local address 127.0.0.1:40834)
0928 14:47:20.604675 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:20.604714 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.124:46023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260928 14:47:20.605212 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.605072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:46023 (local address 127.0.0.1:40840)
0928 14:47:20.605139 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:20.605181 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.33.124:46023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260928 14:47:20.605347 32022 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.33.124:46023: Network error: Client connection negotiation failed: client connection to 127.9.33.124:46023: connect: Connection refused (error 111)
I20260928 14:47:20.606417 32104 rpc_server.cc:307] RPC server started. Bound to: 127.9.33.124:46023
I20260928 14:47:20.606488 32171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.33.124:46023 every 8 connection(s)
I20260928 14:47:20.606858 32104 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-2/data/info.pb
I20260928 14:47:20.607937 32172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260928 14:47:20.609543 32172 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35843 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:47:20.611830 9349 external_mini_cluster.cc:1388] Started /tmp/dist-test-task4TnxN9/build/release/bin/kudu as pid 32104
I20260928 14:47:20.611961 9349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task4TnxN9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790606561545179-9349-0/minicluster-data/master-2/wal/instance
I20260928 14:47:20.612198 9349 external_mini_cluster.cc:1428] Setting key 90927a608cd6374918ada1f5ee485541
Sep 28 14:47:20 dist-test-slave-pv0w krb5kdc[31941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606840, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.124@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
Sep 28 14:47:20 dist-test-slave-pv0w krb5kdc[31941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.124@KRBTEST.COM
I20260928 14:47:20.620313 32025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.610272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50072 (local address 127.9.33.126:35843)
0928 14:47:20.610334 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:20.610336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:20.610559 (+ 223us) server_negotiation.cc:407] Connection header received
0928 14:47:20.611028 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:20.611029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:20.611038 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:20.611094 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:20.611706 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.612374 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.613883 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.614069 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.614129 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:20.616888 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:20.616889 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:20.616890 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:47:20.616897 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:20.618495 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:20.618984 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:20.618986 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:20.618986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:20.619088 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:20.619450 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:20.619451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:20.619451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:20.619700 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:20.619805 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:20.620104 (+ 299us) server_negotiation.cc:299] Negotiation successful
0928 14:47:20.620191 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260928 14:47:20.620389 32174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.610261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.126:35843 (local address 127.0.0.1:50072)
0928 14:47:20.610425 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:20.610468 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0928 14:47:20.610930 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:47:20.611149 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:47:20.611152 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:47:20.611398 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:47:20.611605 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:47:20.611609 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.612552 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.612554 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:47:20.613786 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:47:20.613788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.614174 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.614175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:47:20.614307 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:20.614971 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:47:20.614980 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:47:20.616793 (+ 1813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:47:20.618580 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:47:20.618589 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:47:20.618592 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:47:20.618900 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:47:20.619204 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:47:20.619205 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:47:20.619205 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:47:20.619333 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:47:20.619799 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:47:20.619807 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:47:20.619955 (+ 148us) client_negotiation.cc:769] Sending connection context
0928 14:47:20.620055 (+ 100us) client_negotiation.cc:240] Negotiation successful
0928 14:47:20.620188 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":92,"threads_started":1}
I20260928 14:47:20.621040 32172 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.33.125" port: 45123 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:47:20.621975 32176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.612595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40842 (local address 127.9.33.124:46023)
0928 14:47:20.612764 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:20.612769 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:20.612881 (+ 112us) server_negotiation.cc:407] Connection header received
0928 14:47:20.612904 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:20.612906 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:20.612927 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:20.613047 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:20.613624 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.614626 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.615486 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.615943 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.616008 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:20.618637 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:20.618644 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:20.618645 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:47:20.618651 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:20.620483 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:20.620893 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:20.620896 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:20.620896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:20.620955 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:20.621224 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:20.621225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:20.621226 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:20.621489 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:20.621663 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:20.621759 (+ 96us) server_negotiation.cc:299] Negotiation successful
0928 14:47:20.621861 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":138,"thread_start_us":99,"threads_started":1}
Sep 28 14:47:20 dist-test-slave-pv0w krb5kdc[31941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606840, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.124@KRBTEST.COM for kudu/127.9.33.125@KRBTEST.COM
Sep 28 14:47:20 dist-test-slave-pv0w krb5kdc[31941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.126@KRBTEST.COM
I20260928 14:47:20.628949 32174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.621324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.125:45123 (local address 127.0.0.1:40198)
0928 14:47:20.621376 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:20.621386 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0928 14:47:20.621421 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:47:20.621608 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:47:20.621609 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:47:20.621749 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:47:20.621784 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:47:20.621785 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.622637 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.622639 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:47:20.623198 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:47:20.623200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.623615 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.623616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:47:20.623679 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:20.624180 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:47:20.624185 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:47:20.625461 (+ 1276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:47:20.627364 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:47:20.627366 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:47:20.627366 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:47:20.627670 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:47:20.628192 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:47:20.628193 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:47:20.628194 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:47:20.628276 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:47:20.628636 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:47:20.628637 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:47:20.628688 (+ 51us) client_negotiation.cc:769] Sending connection context
0928 14:47:20.628771 (+ 83us) client_negotiation.cc:240] Negotiation successful
0928 14:47:20.628875 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260928 14:47:20.629058 32101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.621409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40198 (local address 127.9.33.125:45123)
0928 14:47:20.621468 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:20.621470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:20.621478 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:47:20.621503 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:20.621504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:20.621514 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:20.621606 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:20.621870 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.622560 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.623286 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.623530 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.623616 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:20.625557 (+ 1941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:20.625558 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:20.625559 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:47:20.625567 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:20.627283 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:20.628033 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:20.628034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:20.628034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:20.628095 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:20.628355 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:20.628355 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:20.628356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:20.628510 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:20.628643 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:20.628834 (+ 191us) server_negotiation.cc:299] Negotiation successful
0928 14:47:20.628927 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260928 14:47:20.629416 32172 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.33.124" port: 46023 } has no permanent_uuid. Determining permanent_uuid...
I20260928 14:47:20.630849 32025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.622766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50088 (local address 127.9.33.126:35843)
0928 14:47:20.622810 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:20.622812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:20.622822 (+ 10us) server_negotiation.cc:407] Connection header received
0928 14:47:20.622877 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:20.622878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:20.622887 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:20.622941 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:20.623407 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.624012 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.624758 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.624941 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.624997 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:20.627812 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:20.627814 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:20.627814 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:47:20.627821 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:20.629582 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:20.629983 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:20.629984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:20.629984 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:20.630049 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:20.630259 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:20.630260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:20.630260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:20.630393 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:20.630645 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:20.630685 (+ 40us) server_negotiation.cc:299] Negotiation successful
0928 14:47:20.630758 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Sep 28 14:47:20 dist-test-slave-pv0w krb5kdc[31941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606840, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.124@KRBTEST.COM for kudu/127.9.33.124@KRBTEST.COM
Sep 28 14:47:20 dist-test-slave-pv0w krb5kdc[31941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.33.125@KRBTEST.COM
I20260928 14:47:20.637296 32174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.629742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:46023 (local address 127.0.0.1:40846)
0928 14:47:20.629789 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:20.629798 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0928 14:47:20.629836 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:47:20.629941 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:47:20.629942 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:47:20.630073 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:47:20.630137 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:47:20.630139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.631045 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.631046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:47:20.631816 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:47:20.631823 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.632339 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.632340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:47:20.632394 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:20.632765 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:47:20.632771 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:47:20.634040 (+ 1269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:47:20.636118 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:47:20.636119 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:47:20.636120 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:47:20.636288 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:47:20.636497 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:47:20.636497 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:47:20.636498 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:47:20.636553 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:47:20.636875 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:47:20.636876 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:47:20.636916 (+ 40us) client_negotiation.cc:769] Sending connection context
0928 14:47:20.637040 (+ 124us) client_negotiation.cc:240] Negotiation successful
0928 14:47:20.637177 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260928 14:47:20.637307 32176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.629775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40846 (local address 127.9.33.124:46023)
0928 14:47:20.629813 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:20.629815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:20.629824 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:47:20.629887 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:20.629888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:20.629895 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:20.629936 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:20.630213 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.630959 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.631893 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.632253 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.632322 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:20.634132 (+ 1810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:20.634133 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:20.634134 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:47:20.634143 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:20.636023 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:20.636379 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:20.636381 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:20.636381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:20.636433 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:20.636615 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:20.636617 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:20.636617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:20.636773 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:20.636856 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:20.637083 (+ 227us) server_negotiation.cc:299] Negotiation successful
0928 14:47:20.637227 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":74,"server-negotiator.queue_time_us":17}
I20260928 14:47:20.638873 32172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a426bb520f00401d94b1aca8b5c77f30: Bootstrap starting.
I20260928 14:47:20.639868 32172 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a426bb520f00401d94b1aca8b5c77f30: Neither blocks nor log segments found. Creating new log.
I20260928 14:47:20.640146 32101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.631536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40200 (local address 127.9.33.125:45123)
0928 14:47:20.631572 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:20.631573 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:20.631581 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:47:20.631624 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:20.631625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:20.631633 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:20.631687 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:20.632339 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.633084 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.633749 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.634115 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.634168 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:20.636952 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:20.636953 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:20.636953 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:47:20.636961 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:20.638811 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:20.639205 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:20.639207 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:20.639207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:20.639267 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:20.639501 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:20.639502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:20.639503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:20.639626 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:20.639860 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:20.639898 (+ 38us) server_negotiation.cc:299] Negotiation successful
0928 14:47:20.640021 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260928 14:47:20.640208 32172 log.cc:824] T 00000000000000000000000000000000 P a426bb520f00401d94b1aca8b5c77f30: Log is configured to *not* fsync() on all Append() calls
I20260928 14:47:20.641057 32172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a426bb520f00401d94b1aca8b5c77f30: No bootstrap required, opened a new log
I20260928 14:47:20.641605 9349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260928 14:47:20.642967 32172 raft_consensus.cc:348] T 00000000000000000000000000000000 P a426bb520f00401d94b1aca8b5c77f30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e580e5dc9c41a8a8107fd92e860e61" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35843 } } peers { permanent_uuid: "d0edf43b2d484bf08df745bb30eebd3e" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 45123 } } peers { permanent_uuid: "a426bb520f00401d94b1aca8b5c77f30" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 46023 } }
I20260928 14:47:20.643128 32172 raft_consensus.cc:729] T 00000000000000000000000000000000 P a426bb520f00401d94b1aca8b5c77f30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a426bb520f00401d94b1aca8b5c77f30, State: Initialized, Role: FOLLOWER
I20260928 14:47:20.643226 32172 consensus_queue.cc:260] T 00000000000000000000000000000000 P a426bb520f00401d94b1aca8b5c77f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e580e5dc9c41a8a8107fd92e860e61" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35843 } } peers { permanent_uuid: "d0edf43b2d484bf08df745bb30eebd3e" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 45123 } } peers { permanent_uuid: "a426bb520f00401d94b1aca8b5c77f30" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 46023 } }
I20260928 14:47:20.643613 32172 sys_catalog.cc:564] T 00000000000000000000000000000000 P a426bb520f00401d94b1aca8b5c77f30 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:47:20.643546 32178 sys_catalog.cc:455] T 00000000000000000000000000000000 P a426bb520f00401d94b1aca8b5c77f30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e580e5dc9c41a8a8107fd92e860e61" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35843 } } peers { permanent_uuid: "d0edf43b2d484bf08df745bb30eebd3e" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 45123 } } peers { permanent_uuid: "a426bb520f00401d94b1aca8b5c77f30" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 46023 } } }
I20260928 14:47:20.643679 32178 sys_catalog.cc:458] T 00000000000000000000000000000000 P a426bb520f00401d94b1aca8b5c77f30 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:47:20.642892 31996 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b7e580e5dc9c41a8a8107fd92e860e61"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50088
W20260928 14:47:20.645576 32189 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a426bb520f00401d94b1aca8b5c77f30: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:47:20.645639 32189 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:47:20.645802 32104 master_runner.cc:186] Error getting master registration for 127.9.33.126:35843: OK, instance_id { permanent_uuid: "b7e580e5dc9c41a8a8107fd92e860e61" instance_seqno: 1790606840261382 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 14:47:20.648943 32104 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 14:47:20.664615 32097 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.33.124" port: 46023 } attempt: 3
Sep 28 14:47:20 dist-test-slave-pv0w krb5kdc[31941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606840, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.125@KRBTEST.COM for kudu/127.9.33.124@KRBTEST.COM
I20260928 14:47:20.672708 32099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.665043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:46023 (local address 127.0.0.1:40854)
0928 14:47:20.665143 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:20.665163 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0928 14:47:20.665233 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:47:20.665433 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:47:20.665434 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:47:20.665613 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:47:20.665672 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:47:20.665674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.666358 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.666359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:47:20.666899 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:47:20.666901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.667313 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.667314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:47:20.667384 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:20.667915 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:47:20.667924 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:47:20.669635 (+ 1711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:47:20.671500 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:47:20.671501 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:47:20.671501 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:47:20.671690 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:47:20.671932 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:47:20.671932 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:47:20.671933 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:47:20.672019 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:47:20.672343 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:47:20.672344 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:47:20.672396 (+ 52us) client_negotiation.cc:769] Sending connection context
0928 14:47:20.672488 (+ 92us) client_negotiation.cc:240] Negotiation successful
0928 14:47:20.672601 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260928 14:47:20.672708 32176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.665229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40854 (local address 127.9.33.124:46023)
0928 14:47:20.665299 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:20.665301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:20.665310 (+ 9us) server_negotiation.cc:407] Connection header received
0928 14:47:20.665341 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:20.665342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:20.665352 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:20.665427 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:20.665764 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.666263 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.666996 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.667221 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.667312 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:20.669742 (+ 2430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:20.669744 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:20.669744 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:47:20.669753 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:20.671430 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:20.671813 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:20.671815 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:20.671816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:20.671875 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:20.672094 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:20.672095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:20.672095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:20.672261 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:20.672340 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:20.672514 (+ 174us) server_negotiation.cc:299] Negotiation successful
0928 14:47:20.672601 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260928 14:47:20.675009 32097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0edf43b2d484bf08df745bb30eebd3e: Bootstrap starting.
I20260928 14:47:20.675832 32097 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0edf43b2d484bf08df745bb30eebd3e: Neither blocks nor log segments found. Creating new log.
I20260928 14:47:20.676149 32097 log.cc:824] T 00000000000000000000000000000000 P d0edf43b2d484bf08df745bb30eebd3e: Log is configured to *not* fsync() on all Append() calls
I20260928 14:47:20.677096 32097 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0edf43b2d484bf08df745bb30eebd3e: No bootstrap required, opened a new log
I20260928 14:47:20.679126 32097 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0edf43b2d484bf08df745bb30eebd3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e580e5dc9c41a8a8107fd92e860e61" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35843 } } peers { permanent_uuid: "d0edf43b2d484bf08df745bb30eebd3e" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 45123 } } peers { permanent_uuid: "a426bb520f00401d94b1aca8b5c77f30" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 46023 } }
I20260928 14:47:20.679283 32097 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0edf43b2d484bf08df745bb30eebd3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0edf43b2d484bf08df745bb30eebd3e, State: Initialized, Role: FOLLOWER
I20260928 14:47:20.679375 32097 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0edf43b2d484bf08df745bb30eebd3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e580e5dc9c41a8a8107fd92e860e61" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35843 } } peers { permanent_uuid: "d0edf43b2d484bf08df745bb30eebd3e" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 45123 } } peers { permanent_uuid: "a426bb520f00401d94b1aca8b5c77f30" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 46023 } }
I20260928 14:47:20.679663 32192 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0edf43b2d484bf08df745bb30eebd3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e580e5dc9c41a8a8107fd92e860e61" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35843 } } peers { permanent_uuid: "d0edf43b2d484bf08df745bb30eebd3e" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 45123 } } peers { permanent_uuid: "a426bb520f00401d94b1aca8b5c77f30" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 46023 } } }
I20260928 14:47:20.679762 32097 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0edf43b2d484bf08df745bb30eebd3e [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:47:20.679792 32192 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0edf43b2d484bf08df745bb30eebd3e [sys.catalog]: This master's current role is: FOLLOWER
W20260928 14:47:20.681600 32203 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d0edf43b2d484bf08df745bb30eebd3e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:47:20.681715 32203 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:47:20.681813 32029 master_runner.cc:186] Error getting master registration for 127.9.33.126:35843: OK, instance_id { permanent_uuid: "b7e580e5dc9c41a8a8107fd92e860e61" instance_seqno: 1790606840261382 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260928 14:47:20.684672 32029 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 14:47:20.704521 32022 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.33.124" port: 46023 } attempt: 3
Sep 28 14:47:20 dist-test-slave-pv0w krb5kdc[31941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790606840, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.33.126@KRBTEST.COM for kudu/127.9.33.124@KRBTEST.COM
I20260928 14:47:20.713096 32024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.704906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.33.124:46023 (local address 127.0.0.1:40856)
0928 14:47:20.704967 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0928 14:47:20.704989 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0928 14:47:20.705035 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0928 14:47:20.705215 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0928 14:47:20.705216 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0928 14:47:20.705418 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0928 14:47:20.705479 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:47:20.705481 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.706444 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.706445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:47:20.707331 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0928 14:47:20.707334 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.707827 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.707828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0928 14:47:20.707891 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:20.708438 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0928 14:47:20.708444 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0928 14:47:20.710123 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0928 14:47:20.711993 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:47:20.711994 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:47:20.711995 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:47:20.712227 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:47:20.712424 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0928 14:47:20.712425 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0928 14:47:20.712425 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0928 14:47:20.712480 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0928 14:47:20.712756 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0928 14:47:20.712757 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0928 14:47:20.712805 (+ 48us) client_negotiation.cc:769] Sending connection context
0928 14:47:20.712869 (+ 64us) client_negotiation.cc:240] Negotiation successful
0928 14:47:20.712974 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260928 14:47:20.713611 32176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.704978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40856 (local address 127.9.33.124:46023)
0928 14:47:20.705034 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:20.705036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:20.705049 (+ 13us) server_negotiation.cc:407] Connection header received
0928 14:47:20.705086 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:20.705088 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:20.705098 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:20.705180 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:20.705568 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.706340 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.707421 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.707754 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.707829 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:20.710220 (+ 2391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:20.710222 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:20.710222 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:47:20.710230 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:20.711898 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:20.712304 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:20.712305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:20.712306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:20.712361 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:20.712544 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:20.712545 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:20.712546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:20.712675 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:20.712732 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:20.713437 (+ 705us) server_negotiation.cc:299] Negotiation successful
0928 14:47:20.713518 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260928 14:47:20.715613 32022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7e580e5dc9c41a8a8107fd92e860e61: Bootstrap starting.
I20260928 14:47:20.716468 32022 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7e580e5dc9c41a8a8107fd92e860e61: Neither blocks nor log segments found. Creating new log.
I20260928 14:47:20.716784 32022 log.cc:824] T 00000000000000000000000000000000 P b7e580e5dc9c41a8a8107fd92e860e61: Log is configured to *not* fsync() on all Append() calls
I20260928 14:47:20.717729 32022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7e580e5dc9c41a8a8107fd92e860e61: No bootstrap required, opened a new log
I20260928 14:47:20.719169 32022 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7e580e5dc9c41a8a8107fd92e860e61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e580e5dc9c41a8a8107fd92e860e61" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35843 } } peers { permanent_uuid: "d0edf43b2d484bf08df745bb30eebd3e" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 45123 } } peers { permanent_uuid: "a426bb520f00401d94b1aca8b5c77f30" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 46023 } }
I20260928 14:47:20.719322 32022 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7e580e5dc9c41a8a8107fd92e860e61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7e580e5dc9c41a8a8107fd92e860e61, State: Initialized, Role: FOLLOWER
I20260928 14:47:20.719477 32022 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7e580e5dc9c41a8a8107fd92e860e61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e580e5dc9c41a8a8107fd92e860e61" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35843 } } peers { permanent_uuid: "d0edf43b2d484bf08df745bb30eebd3e" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 45123 } } peers { permanent_uuid: "a426bb520f00401d94b1aca8b5c77f30" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 46023 } }
I20260928 14:47:20.719895 32022 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7e580e5dc9c41a8a8107fd92e860e61 [sys.catalog]: configured and running, proceeding with master startup.
I20260928 14:47:20.719954 31996 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7e580e5dc9c41a8a8107fd92e860e61 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260928 14:47:20.720019 31996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7e580e5dc9c41a8a8107fd92e860e61 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:47:20.720824 31996 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7e580e5dc9c41a8a8107fd92e860e61 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e580e5dc9c41a8a8107fd92e860e61" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35843 } } peers { permanent_uuid: "d0edf43b2d484bf08df745bb30eebd3e" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 45123 } } peers { permanent_uuid: "a426bb520f00401d94b1aca8b5c77f30" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 46023 } }
I20260928 14:47:20.721066 31996 leader_election.cc:288] T 00000000000000000000000000000000 P b7e580e5dc9c41a8a8107fd92e860e61 [CANDIDATE]: Term 1 election: Requested vote from peers d0edf43b2d484bf08df745bb30eebd3e (127.9.33.125:45123), a426bb520f00401d94b1aca8b5c77f30 (127.9.33.124:46023)
W20260928 14:47:20.721911 32217 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b7e580e5dc9c41a8a8107fd92e860e61: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260928 14:47:20.722528 32217 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260928 14:47:20.722518 32071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b7e580e5dc9c41a8a8107fd92e860e61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d0edf43b2d484bf08df745bb30eebd3e"
I20260928 14:47:20.722666 32071 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0edf43b2d484bf08df745bb30eebd3e [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:47:20.723925 32071 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d0edf43b2d484bf08df745bb30eebd3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7e580e5dc9c41a8a8107fd92e860e61 in term 1.
I20260928 14:47:20.724385 32146 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b7e580e5dc9c41a8a8107fd92e860e61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a426bb520f00401d94b1aca8b5c77f30"
I20260928 14:47:20.724514 32146 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a426bb520f00401d94b1aca8b5c77f30 [term 0 FOLLOWER]: Advancing to term 1
I20260928 14:47:20.725379 32146 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a426bb520f00401d94b1aca8b5c77f30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7e580e5dc9c41a8a8107fd92e860e61 in term 1.
I20260928 14:47:20.725675 31970 leader_election.cc:302] T 00000000000000000000000000000000 P b7e580e5dc9c41a8a8107fd92e860e61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a426bb520f00401d94b1aca8b5c77f30, b7e580e5dc9c41a8a8107fd92e860e61; no voters:
I20260928 14:47:20.727917 31954 master_runner.cc:263] No leader master found from master b7e580e5dc9c41a8a8107fd92e860e61
I20260928 14:47:20.727972 31954 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260928 14:47:20.719880 32206 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7e580e5dc9c41a8a8107fd92e860e61 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e580e5dc9c41a8a8107fd92e860e61" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35843 } } peers { permanent_uuid: "d0edf43b2d484bf08df745bb30eebd3e" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 45123 } } peers { permanent_uuid: "a426bb520f00401d94b1aca8b5c77f30" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 46023 } } }
I20260928 14:47:20.728171 32206 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7e580e5dc9c41a8a8107fd92e860e61 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:47:20.728267 32206 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7e580e5dc9c41a8a8107fd92e860e61 [term 1 FOLLOWER]: Leader election won for term 1
I20260928 14:47:20.728353 32206 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7e580e5dc9c41a8a8107fd92e860e61 [term 1 LEADER]: Becoming Leader. State: Replica: b7e580e5dc9c41a8a8107fd92e860e61, State: Running, Role: LEADER
I20260928 14:47:20.728471 32206 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7e580e5dc9c41a8a8107fd92e860e61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e580e5dc9c41a8a8107fd92e860e61" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35843 } } peers { permanent_uuid: "d0edf43b2d484bf08df745bb30eebd3e" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 45123 } } peers { permanent_uuid: "a426bb520f00401d94b1aca8b5c77f30" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 46023 } }
I20260928 14:47:20.729050 32206 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7e580e5dc9c41a8a8107fd92e860e61 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7e580e5dc9c41a8a8107fd92e860e61. Latest consensus state: current_term: 1 leader_uuid: "b7e580e5dc9c41a8a8107fd92e860e61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e580e5dc9c41a8a8107fd92e860e61" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35843 } } peers { permanent_uuid: "d0edf43b2d484bf08df745bb30eebd3e" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 45123 } } peers { permanent_uuid: "a426bb520f00401d94b1aca8b5c77f30" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 46023 } } }
I20260928 14:47:20.729171 32206 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7e580e5dc9c41a8a8107fd92e860e61 [sys.catalog]: This master's current role is: LEADER
I20260928 14:47:20.730592 32225 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260928 14:47:20.730852 32225 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260928 14:47:20.738945 32146 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a426bb520f00401d94b1aca8b5c77f30 [term 1 FOLLOWER]: Refusing update from remote peer b7e580e5dc9c41a8a8107fd92e860e61: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:47:20.739287 32071 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d0edf43b2d484bf08df745bb30eebd3e [term 1 FOLLOWER]: Refusing update from remote peer b7e580e5dc9c41a8a8107fd92e860e61: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260928 14:47:20.739737 32223 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b7e580e5dc9c41a8a8107fd92e860e61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a426bb520f00401d94b1aca8b5c77f30" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 46023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:47:20.739959 32223 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b7e580e5dc9c41a8a8107fd92e860e61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0edf43b2d484bf08df745bb30eebd3e" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 45123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260928 14:47:20.741647 32192 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0edf43b2d484bf08df745bb30eebd3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7e580e5dc9c41a8a8107fd92e860e61. Latest consensus state: current_term: 1 leader_uuid: "b7e580e5dc9c41a8a8107fd92e860e61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e580e5dc9c41a8a8107fd92e860e61" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35843 } } peers { permanent_uuid: "d0edf43b2d484bf08df745bb30eebd3e" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 45123 } } peers { permanent_uuid: "a426bb520f00401d94b1aca8b5c77f30" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 46023 } } }
I20260928 14:47:20.741748 32192 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0edf43b2d484bf08df745bb30eebd3e [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:47:20.742538 32178 sys_catalog.cc:455] T 00000000000000000000000000000000 P a426bb520f00401d94b1aca8b5c77f30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7e580e5dc9c41a8a8107fd92e860e61. Latest consensus state: current_term: 1 leader_uuid: "b7e580e5dc9c41a8a8107fd92e860e61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e580e5dc9c41a8a8107fd92e860e61" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35843 } } peers { permanent_uuid: "d0edf43b2d484bf08df745bb30eebd3e" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 45123 } } peers { permanent_uuid: "a426bb520f00401d94b1aca8b5c77f30" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 46023 } } }
I20260928 14:47:20.742654 32178 sys_catalog.cc:458] T 00000000000000000000000000000000 P a426bb520f00401d94b1aca8b5c77f30 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:47:20.753005 32206 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7e580e5dc9c41a8a8107fd92e860e61 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b7e580e5dc9c41a8a8107fd92e860e61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e580e5dc9c41a8a8107fd92e860e61" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35843 } } peers { permanent_uuid: "d0edf43b2d484bf08df745bb30eebd3e" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 45123 } } peers { permanent_uuid: "a426bb520f00401d94b1aca8b5c77f30" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 46023 } } }
I20260928 14:47:20.753017 32192 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0edf43b2d484bf08df745bb30eebd3e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b7e580e5dc9c41a8a8107fd92e860e61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e580e5dc9c41a8a8107fd92e860e61" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35843 } } peers { permanent_uuid: "d0edf43b2d484bf08df745bb30eebd3e" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 45123 } } peers { permanent_uuid: "a426bb520f00401d94b1aca8b5c77f30" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 46023 } } }
I20260928 14:47:20.753132 32192 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0edf43b2d484bf08df745bb30eebd3e [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:47:20.753132 32206 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7e580e5dc9c41a8a8107fd92e860e61 [sys.catalog]: This master's current role is: LEADER
I20260928 14:47:20.753180 32206 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7e580e5dc9c41a8a8107fd92e860e61 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b7e580e5dc9c41a8a8107fd92e860e61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e580e5dc9c41a8a8107fd92e860e61" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35843 } } peers { permanent_uuid: "d0edf43b2d484bf08df745bb30eebd3e" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 45123 } } peers { permanent_uuid: "a426bb520f00401d94b1aca8b5c77f30" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 46023 } } }
I20260928 14:47:20.753224 32206 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7e580e5dc9c41a8a8107fd92e860e61 [sys.catalog]: This master's current role is: LEADER
I20260928 14:47:20.753754 32178 sys_catalog.cc:455] T 00000000000000000000000000000000 P a426bb520f00401d94b1aca8b5c77f30 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b7e580e5dc9c41a8a8107fd92e860e61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7e580e5dc9c41a8a8107fd92e860e61" member_type: VOTER last_known_addr { host: "127.9.33.126" port: 35843 } } peers { permanent_uuid: "d0edf43b2d484bf08df745bb30eebd3e" member_type: VOTER last_known_addr { host: "127.9.33.125" port: 45123 } } peers { permanent_uuid: "a426bb520f00401d94b1aca8b5c77f30" member_type: VOTER last_known_addr { host: "127.9.33.124" port: 46023 } } }
I20260928 14:47:20.753858 32178 sys_catalog.cc:458] T 00000000000000000000000000000000 P a426bb520f00401d94b1aca8b5c77f30 [sys.catalog]: This master's current role is: FOLLOWER
I20260928 14:47:20.754124 32232 mvcc.cc:205] Tried to move back new op lower bound from 7334325619662647296 to 7334325619624923136. Current Snapshot: MvccSnapshot[applied={T|T < 7334325619662647296}]
I20260928 14:47:20.754748 32230 mvcc.cc:205] Tried to move back new op lower bound from 7334325619662647296 to 7334325619624923136. Current Snapshot: MvccSnapshot[applied={T|T < 7334325619662647296}]
I20260928 14:47:20.755605 32101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.730006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40204 (local address 127.9.33.125:45123)
0928 14:47:20.730046 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:20.730048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:20.732451 (+ 2403us) server_negotiation.cc:407] Connection header received
0928 14:47:20.732723 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:20.732725 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:20.732737 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:20.732787 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:20.735406 (+ 2619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.736191 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.741177 (+ 4986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.741470 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.741518 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:20.746843 (+ 5325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:20.746845 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:20.746845 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:47:20.746854 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:20.748622 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:20.751217 (+ 2595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:20.751219 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:20.751220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:20.751285 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:20.754270 (+ 2985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:20.754272 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:20.754272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:20.754426 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:20.754512 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:20.755394 (+ 882us) server_negotiation.cc:299] Negotiation successful
0928 14:47:20.755516 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 14:47:20.755898 32225 catalog_manager.cc:1347] Generated new cluster ID: 5a9aa8db54c14b56b380450570d20378
I20260928 14:47:20.755964 32225 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260928 14:47:20.763446 32025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.729602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50098 (local address 127.9.33.126:35843)
0928 14:47:20.729659 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:20.729661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:20.732349 (+ 2688us) server_negotiation.cc:407] Connection header received
0928 14:47:20.732608 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:20.732609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:20.732620 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:20.732653 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:20.737256 (+ 4603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.737934 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.740720 (+ 2786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.741021 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.741072 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:20.749077 (+ 8005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:20.749080 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:20.749080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:47:20.749089 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:20.750787 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:20.753953 (+ 3166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:20.753955 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:20.753955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:20.754032 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:20.754928 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:20.754929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:20.754930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:20.755088 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:20.755173 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:20.763234 (+ 8061us) server_negotiation.cc:299] Negotiation successful
0928 14:47:20.763341 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260928 14:47:20.764971 32176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:20.730334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40864 (local address 127.9.33.124:46023)
0928 14:47:20.730373 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:20.730375 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:20.734500 (+ 4125us) server_negotiation.cc:407] Connection header received
0928 14:47:20.734599 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:20.734600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:20.734612 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:20.734642 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:20.738910 (+ 4268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.739541 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.743834 (+ 4293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:20.744140 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:20.744189 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:20.751392 (+ 7203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0928 14:47:20.751394 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0928 14:47:20.751394 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0928 14:47:20.751403 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0928 14:47:20.753380 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:20.762035 (+ 8655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:20.762037 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:20.762038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:20.762141 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0928 14:47:20.764298 (+ 2157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0928 14:47:20.764300 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0928 14:47:20.764300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0928 14:47:20.764460 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0928 14:47:20.764552 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:20.764792 (+ 240us) server_negotiation.cc:299] Negotiation successful
0928 14:47:20.764881 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 14:47:20.767870 32225 catalog_manager.cc:1370] Generated new certificate authority record
I20260928 14:47:20.768690 32225 catalog_manager.cc:1504] Loading token signing keys...
I20260928 14:47:20.773741 32225 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7e580e5dc9c41a8a8107fd92e860e61: Generated new TSK 0
I20260928 14:47:20.774020 32225 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260928 14:47:20.797545 32025 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:47:20.795746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50102 (local address 127.9.33.126:35843)
0928 14:47:20.795796 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:20.795799 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:20.796763 (+ 964us) server_negotiation.cc:407] Connection header received
0928 14:47:20.796874 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:20.796876 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:20.796888 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:20.796922 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:20.797461 (+ 539us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50102: BlockingRecv error: recv got EOF from 127.0.0.1:50102 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260928 14:47:20.798434 32101 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:47:20.796185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40212 (local address 127.9.33.125:45123)
0928 14:47:20.796230 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:20.796231 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:20.797885 (+ 1654us) server_negotiation.cc:407] Connection header received
0928 14:47:20.798003 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:20.798004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:20.798016 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:20.798046 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:20.798371 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40212: BlockingRecv error: recv got EOF from 127.0.0.1:40212 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260928 14:47:20.799296 32176 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:47:20.796498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40872 (local address 127.9.33.124:46023)
0928 14:47:20.796540 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:20.796542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:20.798724 (+ 2182us) server_negotiation.cc:407] Connection header received
0928 14:47:20.798866 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:20.798867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:20.798877 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:20.798908 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:20.799237 (+ 329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40872: BlockingRecv error: recv got EOF from 127.0.0.1:40872 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260928 14:47:20.801815 32101 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:47:20.801081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40220 (local address 127.9.33.125:45123)
0928 14:47:20.801119 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:20.801121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:20.801298 (+ 177us) server_negotiation.cc:407] Connection header received
0928 14:47:20.801401 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:20.801402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:20.801420 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:20.801449 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:20.801765 (+ 316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40220: BlockingRecv error: recv got EOF from 127.0.0.1:40220 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260928 14:47:20.805063 32101 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:47:20.804268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40234 (local address 127.9.33.125:45123)
0928 14:47:20.804305 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:20.804306 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:20.804533 (+ 227us) server_negotiation.cc:407] Connection header received
0928 14:47:20.804631 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:20.804632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:20.804642 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:20.804672 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:20.805013 (+ 341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40234: BlockingRecv error: recv got EOF from 127.0.0.1:40234 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260928 14:47:20.809299 32101 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:47:20.808458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40236 (local address 127.9.33.125:45123)
0928 14:47:20.808498 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:20.808500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:20.808729 (+ 229us) server_negotiation.cc:407] Connection header received
0928 14:47:20.808831 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:20.808832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:20.808843 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:20.808875 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:20.809245 (+ 370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40236: BlockingRecv error: recv got EOF from 127.0.0.1:40236 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260928 14:47:20.815496 32101 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:47:20.814665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40238 (local address 127.9.33.125:45123)
0928 14:47:20.814707 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:20.814709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:20.814921 (+ 212us) server_negotiation.cc:407] Connection header received
0928 14:47:20.815033 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:20.815034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:20.815045 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:20.815073 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:20.815421 (+ 348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40238: BlockingRecv error: recv got EOF from 127.0.0.1:40238 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260928 14:47:20.825767 32101 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:47:20.824953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40246 (local address 127.9.33.125:45123)
0928 14:47:20.824998 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:20.825000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:20.825229 (+ 229us) server_negotiation.cc:407] Connection header received
0928 14:47:20.825327 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:20.825327 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:20.825338 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:20.825366 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:20.825706 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40246: BlockingRecv error: recv got EOF from 127.0.0.1:40246 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260928 14:47:20.844125 32101 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:47:20.843200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40254 (local address 127.9.33.125:45123)
0928 14:47:20.843249 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:20.843251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:20.843392 (+ 141us) server_negotiation.cc:407] Connection header received
0928 14:47:20.843529 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:20.843530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:20.843549 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:20.843614 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:20.844059 (+ 445us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40254: BlockingRecv error: recv got EOF from 127.0.0.1:40254 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260928 14:47:20.878137 32101 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:47:20.877311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40270 (local address 127.9.33.125:45123)
0928 14:47:20.877378 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:20.877380 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:20.877522 (+ 142us) server_negotiation.cc:407] Connection header received
0928 14:47:20.877643 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:20.877644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:20.877656 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:20.877722 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:20.878072 (+ 350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40270: BlockingRecv error: recv got EOF from 127.0.0.1:40270 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260928 14:47:20.944411 32101 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:47:20.943527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40284 (local address 127.9.33.125:45123)
0928 14:47:20.943656 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:20.943659 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:20.943780 (+ 121us) server_negotiation.cc:407] Connection header received
0928 14:47:20.943878 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:20.943879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:20.943891 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:20.943933 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:20.944332 (+ 399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40284: BlockingRecv error: recv got EOF from 127.0.0.1:40284 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
W20260928 14:47:21.074592 32101 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:47:21.073727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40296 (local address 127.9.33.125:45123)
0928 14:47:21.073787 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:21.073789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:21.073932 (+ 143us) server_negotiation.cc:407] Connection header received
0928 14:47:21.073975 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:21.073976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:21.073988 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:21.074046 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:21.074510 (+ 464us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40296: BlockingRecv error: recv got EOF from 127.0.0.1:40296 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260928 14:47:21.332751 32101 negotiation.cc:337] Failed RPC negotiation. Trace:
0928 14:47:21.331975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40302 (local address 127.9.33.125:45123)
0928 14:47:21.332065 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:21.332067 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:21.332129 (+ 62us) server_negotiation.cc:407] Connection header received
0928 14:47:21.332185 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:21.332186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:21.332197 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:21.332266 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0928 14:47:21.332664 (+ 398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40302: BlockingRecv error: recv got EOF from 127.0.0.1:40302 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260928 14:47:21.646139 32189 catalog_manager.cc:1259] Loaded cluster ID: 5a9aa8db54c14b56b380450570d20378
I20260928 14:47:21.646242 32189 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a426bb520f00401d94b1aca8b5c77f30: loading cluster ID for follower catalog manager: success
I20260928 14:47:21.647049 32189 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a426bb520f00401d94b1aca8b5c77f30: acquiring CA information for follower catalog manager: success
I20260928 14:47:21.647310 32189 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a426bb520f00401d94b1aca8b5c77f30: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 14:47:21.682250 32203 catalog_manager.cc:1259] Loaded cluster ID: 5a9aa8db54c14b56b380450570d20378
I20260928 14:47:21.682340 32203 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d0edf43b2d484bf08df745bb30eebd3e: loading cluster ID for follower catalog manager: success
I20260928 14:47:21.683179 32203 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d0edf43b2d484bf08df745bb30eebd3e: acquiring CA information for follower catalog manager: success
I20260928 14:47:21.683502 32203 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d0edf43b2d484bf08df745bb30eebd3e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260928 14:47:21.848935 32302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:21.846090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40306 (local address 127.9.33.125:45123)
0928 14:47:21.846232 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:21.846234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:21.846290 (+ 56us) server_negotiation.cc:407] Connection header received
0928 14:47:21.846343 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:21.846343 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:21.846398 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:21.846473 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:47:21.846618 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:21.847121 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:21.847966 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:21.848176 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:21.848234 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:21.848516 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:47:21.848641 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:47:21.848717 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:21.848792 (+ 75us) server_negotiation.cc:299] Negotiation successful
0928 14:47:21.848813 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":60,"threads_started":1}
I20260928 14:47:21.853170 32308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:21.850693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40880 (local address 127.9.33.124:46023)
0928 14:47:21.850830 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:21.850832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:21.850853 (+ 21us) server_negotiation.cc:407] Connection header received
0928 14:47:21.850911 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:21.850912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:21.850948 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:21.851013 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:47:21.851113 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:21.851626 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:21.852412 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:21.852651 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:21.852707 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:21.852835 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:47:21.852938 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:47:21.853001 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:21.853037 (+ 36us) server_negotiation.cc:299] Negotiation successful
0928 14:47:21.853054 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":83,"threads_started":1}
I20260928 14:47:21.858359 32302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:21.854960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40308 (local address 127.9.33.125:45123)
0928 14:47:21.855042 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:21.855044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:21.855143 (+ 99us) server_negotiation.cc:407] Connection header received
0928 14:47:21.855236 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:21.855237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:21.855251 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:21.855315 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:47:21.855569 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:21.856218 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:21.857114 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:21.857415 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:21.857493 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:21.858065 (+ 572us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:47:21.858110 (+ 45us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:47:21.858180 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:21.858243 (+ 63us) server_negotiation.cc:299] Negotiation successful
0928 14:47:21.858269 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260928 14:47:21.858359 32308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:21.855251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40892 (local address 127.9.33.124:46023)
0928 14:47:21.855292 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:21.855293 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:21.855518 (+ 225us) server_negotiation.cc:407] Connection header received
0928 14:47:21.855576 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:21.855577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:21.855590 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:21.855646 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:47:21.855780 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:21.856519 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:21.857511 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:21.857801 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:21.857866 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:21.858086 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:47:21.858120 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:47:21.858149 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:21.858250 (+ 101us) server_negotiation.cc:299] Negotiation successful
0928 14:47:21.858274 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260928 14:47:21.858713 32315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0928 14:47:21.855138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50114 (local address 127.9.33.126:35843)
0928 14:47:21.855619 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0928 14:47:21.855621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0928 14:47:21.855629 (+ 8us) server_negotiation.cc:407] Connection header received
0928 14:47:21.855674 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0928 14:47:21.855675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0928 14:47:21.855750 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0928 14:47:21.855833 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0928 14:47:21.855914 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:21.856659 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:21.857542 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0928 14:47:21.857863 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0928 14:47:21.857926 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0928 14:47:21.858133 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0928 14:47:21.858258 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0928 14:47:21.858311 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0928 14:47:21.858608 (+ 297us) server_negotiation.cc:299] Negotiation successful
0928 14:47:21.858638 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":58,"threads_started":1}
I20260928 14:47:21.866935 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 31954
I20260928 14:47:21.872409 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 32029
I20260928 14:47:21.877643 9349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task4TnxN9/build/release/bin/kudu with pid 32104
2026-09-28T14:47:21Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2922 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5855 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (280340 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST